StreamBase New and Noteworthy

Date: 11 Feb 2016

This topic summarizes the history of significant changes in TIBCO StreamBase® releases. For current and resolved product limitations, see the StreamBase Release Notes.

New in StreamBase 7.6.2

Fixes

TIBCO StreamBase® 7.6.2 fixes several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

Updates from Previous Releases

TIBCO StreamBase® 7.6.2 inherits the updates and new features added to maintenance releases of previous release series, through and including release 7.5.5.

New and Updated Features

New Map Operators Added for TIBCO FTL Adapters

The TIBCO StreamBase adapter solution for TIBCO FTL® was enhanced with four new operators that provide support for the FTL Map operation feature that was added to TIBCO FTL in its release 4.2 and later. These operators allow a StreamBase application to perform various operations on FTL maps, including getting and setting messages, iterating through a map's messages, and removing messages from a map. The four FTL Map operators are described on TIBCO FTL Map Adapters in the Adapters Guide. The new operators are illustrated in a new sample added to the existing TIBCO FTL samples described in the Samples Guide.

ActiveSpaces Adapter Updated

ActiveSpaces operators that provide a status port were updated to provide better feedback when a connection is lost. The operators now include a Connect On Startup adapter property (set to true by default to preserve existing functionality). If set to False, the node does not try to connect to the Metaspace. Instead, it relies on whatever other nodes connecting to the same space to establish the connection.

Any ActiveSpaces operator with a status port now emits status tuples on detecting any ActiveSpaces state change (such as INITIAL to READY), and on detecting any member joining or leaving the space.

TERR Operator Updated

The TERR Operator was updated to support an optional command input port as well as a data input port. The command port supports a single command string, reset, which causes the current TERR instance to be stopped and restarted.

The operator now supports specifying whether the script entry to monitor is a directory or file. If you specify a directory, only files with the .RData extensions (with that exact spelling) are monitored; other file types in the specified directory that change, are added, or deleted are ignored.

The operator's dynamic and static modes of operation were updated and clarified. For details on these and the above changes, see Using the TERR Operator in the Authoring Guide.

UDP Adapters Updated

The TIBCO StreamBase® Input Adapter for UDP has a new property to specify the maximum packet size. Both the UDP Input Adapter and the Output Adapter for UDP now have the ability to select between unicast and multicast modes.

API Change: ResourceFilePropertyDescriptor Class Updated

The ResourceFilePropertyDescriptor class now allows an operator's Properties view, when prompting for a resource, to select a directory as well as a file.

New in StreamBase 7.6.1

Fixes

TIBCO StreamBase® 7.6.1 fixed several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

Updates from Previous Releases

TIBCO StreamBase® 7.6.1 inherited the updates and new features added to maintenance releases of previous release series, through and including release 7.5.5.

StreamBase Platform Updates

Installation Slowness on Mac OS X Resolved

In previous releases, on any Mac with OS X Yosemite or later, double-clicking one of the DMG distribution files that contains StreamBase plus Live Datamart, or LiveView Desktop, could start a spinning beach ball timeout period that could last for many minutes. An alternate installation method was described in the Installation Guide.

Starting with releases 7.5.5 and 7.6.1, TIBCO StreamBase DMG files now have a workaround for the Apple bug that was creating this situation. Thus, this issue is now completely resolved, and no longer requires the alternate installation method.

New and Updated Features

StreamBase release 7.6.1 added the following updates and new features:

Wording Change for Edit Dimension Dialog

In previous releases, the last line of the Aggregate operator's Edit Dimension dialog, when selecting a Predicate type dimension, was: Evaluate all Emit and Close predicates before adding the new tuple to the window. While this description was technically correct, and the dialog was and is behaving as designed, the wording of this check box's label was causing confusion. As of this release, the label for the final check box in that dialog is now: Do not include the current tuple in the output of an aggregate window when emitted.

New Keyboard Shortcut

A new keyboard shortcut, A A, is available to open the Adapters, Global Operators dialog over the EventFlow Editor canvas.

replaceelement Function Extended

The replaceelement() function in the expression language was extended with two new syntax forms that let you specify an update function or a predicate function instead of an index value for locating the list elements to be replaced.

Dynamic Variables Now Inheritable

The EventFlow Editor's Annotations tab gained a new check box in the Module Annotations section in release 7.6.0. As of 7.6.1, that check box's label now reads Constants and dynamic variables in this module will be visible to any module referenced inside it. This change adds dynamic variables to the support for inherting constants that was added in 7.6.0.

The default state of this check box is cleared. In this state, constants and dynamic variables defined on the Definitions tab have scope only for this module, and can override only parameters defined in this module. With the check box selected, constants and dynamic variables defined in an outer module can override parameters defined in inner modules.

New Profiling Statistics Stream for Optional Use

Customers who use the profiling capabilities of StreamBase are aware that they connect to a system stream named system.statv2. Starting with this release, there is a new profiling stream available, system.statv3. However, the existing statv2 stream is still the default, so no behavior changes are required. By default, all current profiling applications are expected to continue working normally.

For those who wish to experiment with the new statistics stream, you can expect around a ten times improvement in network bandwidth usage, especially with large Live Datamart projects. This improvement comes at the expense of incompatible statistics output: the new stream does not emit human-readable statistics like before. To interpret the new stream, you must write a monitoring application using the StreamBaseMonitor API for either Java or .NET. You can configure StreamBase to emit statistics on the statv3 stream by setting the Java system property streambase.appgen.stats.schema.version=3 or the environment variable STREAMBASE_APPGEN_STATS_SCHEMA_VERSION=3.

New Apache Kafka Adapters

This release includes a new set of adapters for the Apache Kafka publish-subscribe message broker system, in which messaging is implemented as a distributed commit log. The adapters in this suite are implemented as StreamBase global Java operators, as follows:

A sample illustrates the use of these adapters, as described in Apache Kafka Adapter Samples.

TERR Adapter Updated

The TERR adapter was updated to support a new mode of operation where, rather than doing text substitution of variables into your TERR script, the input variables are directly inserted into the TERR environment where they can be accessed by the TERR script. This is expected to increase performance for large input scripts. The original method is still available in the adapter.

TBF Adapter Updated

The Toronto Broadcast Feed Adapter was updated to support 24-digit timestamps in STAMP feeds. The adapter now accepts either 17-digit or 24-digit STAMP-format timestamps.

Feed Simulation Adapter Updated

The StreamBase Feed Simulation adapter was extended to accept a file name as an optional third argument on its control port. This allows you to apply that adapter's feed simulation to more than one file, changeable at runtime. The specified file must exist, and it must match the same output schema as the file currently running, and must have access to all resources required.

Web Server Response Adapter Updated

The Web Server Response Adapter was updated with the ability to add default HTTP headers to the response. These default headers are applied to each HTTP response, but if the input tuple contains HTTP headers, they override the defaults.

JDK and Java Version Changes

On all platforms, the 64-bit Oracle JDK version bundled with StreamBase for private use by StreamBase and Live Datamart applications is now 1.8.0_u65. JDK 8 is required to run StreamBase Server and StreamBase Studio.

The embedded JDK is used to launch StreamBase Server, StreamBase Studio, LiveView Server, and all StreamBase and Live Datamart applications. As before, you can override this installed default to use a different supported JDK, as described in Using an External JDK.

The Java Client Libraries for StreamBase and LiveView are fully compatible with Java 7, but we recommend using Oracle Java 8, preferably 1.8.0_u65 or later, to compile and run your own client applications, custom functions, custom adapters, and monitoring applications. The Java Client Libraries themselves are compiled with target set to Java 1.7 to maintain compatibility.

As before, TIBCO supports and recommends a customer-installed instance of Azul Zing for deployments of TIBCO LiveView servers on Linux x86_64 platforms when very large heap sizes are required. Contact Azul Systems to obtain and license Zing for LiveView. When Zing is installed in its default location, LiveView Server detects and launches with it by default. TIBCO LiveView servers support Azul Zing versions that correspond to Oracle JDK 8 versions.

Java API Update

In the StreamBaseMonitor API, the OperatorInfo#getPercentUsage method was changed to return the exponentially weighted moving average of the current CPU usage. This brings the API in line with the sbmonitor command line tool, with always reported EWMA.

Documentation on the Streams Profiling Output

The Profiling page in the StreamBase Administration Guide was updated to discuss statistics emitted by the sbprofile command for input and output streams.

New in StreamBase 7.6.0

Fixes

TIBCO StreamBase® 7.6.0 fixed several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

Updates from Previous Releases

TIBCO StreamBase® 7.6.0 inherited the updates and new features added to maintenance releases of previous release series, through and including releases 7.5.4 and 7.4.5.

StreamBase and Live Datamart — Integrated Installer

TIBCO Live Datamart 2.1 is automatically installed as part of TIBCO StreamBase® 7.6. TIBCO StreamBase and TIBCO Live Datamart are separately licensed, as described on TIBCO StreamBase License Considerations. However, you no longer need to download and run a separate installer to access LiveView features.

For further details on the New and Updated features of TIBCO Live Datamart, see the separate New and Noteworthy for the Live Datamart product family.

Updates to Packaging

Connectivity Package Updates

The TIBCO StreamBase CEP Connectivity Package contained several premium adapters that have separate licensing concerns, and are not included in the base TIBCO StreamBase installation. If you licensed any of these adapters, install the StreamBase Connectivity Package from the TIBCO download site.

The contents of this package are subject to change from release to release. Starting with releases 7.5.5 and 7.6.0, the following adapter is no longer part of either the Connectivity Package or the TIBCO StreamBase base kit:

  • The TIBCO StreamBase® Adapter for FXall Relationship Trading Customer

New and Updated Features

TIBCO StreamBase® 7.6.0 added the updates and new features shown in the following sections.

 Studio: New Features and Updates

Updates for Load StreamBase Samples Dialog
  • The Load StreamBase Samples dialog now provides a new check box Open LiveView Project Viewer. Select this option when opening a TIBCO Live Datamart sample. The dialog remembers the state of this check box between sessions.

  • The dialog is now resizable.

Updates for Insert Operator or Adapter Dialog

The Insert an Operator or Adapter dialog (the one that opens when you drag the Global Operators, Adapters icon from the Palette view to the canvas) no longer shows the fully qualified path name of the class that implements each operator or adapter. When an item is selected, description text shows in the area below the title, above the OK button.

This change means that the formerly documented method of showing only Global Java Operators in this dialog no longer works. Instead, use the O, V keystroke combination for this purpose.

Update for Initial, Blank Property View

When you first open Studio with an empty workspace and no previous use, when you exit the Welcome screen and click the Go to Authoring link, the initial SB Authoring perspective shows a number of starting point links in the empty Property view. As of this release, a new link was added to help you import projects from the workspace of earlier releases.

Data Construct Cosmetic Changes

The EventFlow Editor canvas rendering of data constructs, including Query Tables, is now a fixed size diamond, with the operator's name appearing in a caption underneath. The size of the diamond no longer expands to fit the table name.

Icon Decoration for Query Table Initial Contents

Query Tables that load initial contents at startup of their containing module now show a red asterisk overlay:

Changes for Query Table Delta Streams

Query Table delta streams are now enabled with an option setting on the Table Settings tab in the Properties view. The context menu option to enable a delta stream is now removed. If a module that has a delta stream configured in a previous release is loaded into 7.6+, the functionality remains as it was in the previous release, but the context menu now includes an option to upgrade the delta stream to the new delta stream format. Schema changes for the Query Table are now automatically propagated to delta streams.

New Import from XSD Menu Option, Definitions Tab

In the Definitions tab of the EventFlow Editor, the context menu in the Constants, Named Schemas, and Table Schemas section has a new option, Import Schemas from XSD. This allows you to generate Named or Table schemas from existing XSD or XML definitions.

New Feature in Annotations Tab

The EventFlow Editor's Annotations tab has a new check box in the Module Annotations section: Constants in this module will be visible to any module referenced inside it. The default state of this check box is cleared. In this state, constants defined on the Definitions tab have scope only for this module, and can override only parameters defined in this module. With the check box selected, constants defined in an outer module can override parameters defined in inner modules.

Reuters and FIX Designers No Longer Open by Default

The FIX Schema Designer and Thomson Reuters Schema Designer views are no longer open in the default SB Authoring perspective. Open these views as needed with WindowShow View. If you need these views open every time, consider creating a customized perspective.

 EventFlow Language Updates 

Constants Used as Module Names

You can now use a constant, defined on the EventFlow Editor's Definitions tab, as a module name in an Extension Point operator.

Metronome Operator Updated

The Metronome operator was updated to be able to accept the name of a dynamic variable defined in the same module as the value for the Tuple output interval field in the operator's properties. This allows you to control the emission interval of the operator at runtime. If you define the input field of the variable's associated Input Stream as a double, you can control the tick interval in fractions of a second. When you send in a new tick interval through the dynamic variable, the interval currently underway must complete before the new interval takes over. Thus, a new interval setting of 5 received at the 3rd second of a current 10-second interval must wait 7 seconds before the 5 interval takes over.

Note that if you enter an invalid interval number at runtime, such as -1, tuples are still emitted at the previously set interval, but they now contain the text of the exception from the operator. Best practice is to validate interval-setting entries at runtime to make sure they are valid values and within practical limits.

Aggregate Operator Updated

The Aggregate operator's Add or Edit Dimension dialog, when the dimension type is Field, now includes subfields of a input tuple's nested schema in the Field drop-down list.

Module Reference Updated

The Properties view for Module References now has a new Constants tab. Use this tab to specify constants that can be used to override module parameters.

 Server Configuration Changes 

New sbconf Option to Allow Capture Field in Unused Module Input Stream

StreamBase's default behavior is to fail typechecking when an unbound capture is detected. An unbound capture occurs when an input stream of a module is implemented with a capture field and that input stream is not connected to an input port of the calling Module Reference in the parent application. This situation results in an error because there is no schema to examine or fields to capture for the unbound capture instance.

This existing behavior is preserved in release 7.6.0 to maintain compatibility with previous releases. However, starting with this release, when you are certain your unbound captures are benign, you can optionally suppress the resulting typecheck errors. To do this, use a new option in the Server configuration file syntax: <global><ignore-unbound-captures/></global>.

New TimeService sbconf Options

See the discussion of the TimeService API changes below for news of the new <time-service-configurations> element.

 Expression Language Updates 

The following functions were extended or added to the StreamBase expression language in release 7.6.0:

Extensions for the Function Data Type

The function data type was extended to support the following new features described in Function Data Type:

  • You can have multiple expressions inside a function body, optionally separated by one (or more) semicolons.

  • You can define locally scoped variables with @define(name as expr).

  • You can annotate a function as @cacheable.

Extensions for JSON Parsing Functions

The parsejson() and parsejson_loose() functions now have an optional third argument used to specify a timestamp format with which to parse the JSON string, using format patterns in the manner of java.text.SimpleDateFormat. In the absence of a third argument, timestamps are parsed using the default format string yyyy-MM-dd HH:mm:ss.SSSZ.

Similarly, the tojson() function now has an optional third argument to specify a timestamp format with which to parse timestamp fields in the incoming tuple. When specifying the third argument, the verbosity second argument is no longer optional, and must be explicitly set.

Various New Simple Functions
New Simple Functions for Lists
New Simple Functions for Aggregate Window Information
New Aggregate Functions
New Special Purpose Simple Functions
New Simple Functions Related to the TimeService API

 Command Line Utility Updates 

Updates for the sbc Command

The sbc command now accepts the name of a named schema in the --where clause of a dequeue command.

The sbc and jsbc commands were updated to emit the same results given inputs of NaN, Infinity, and -Infinity to an input stream field of type double. These values are rejected from fields of type int and long.

Filtering Options Removed from sbprofile

The filter options (-f, -Q, -t, -Y, -C, -S, -I, and -O) were removed from the sbprofile command.

New jsbadmin Commands

Two new jsbadmin commands, getTargetTime and fastForwardTime, can retrieve the current time and, in a controlled time environment, advance the current time by a specified number of milliseconds. The new command are described on the jsbadmin reference page; see also the new Using the StreamBase JUnit TimeService page in the Test/Debug Guide.

New sbadmin Option

The sbadmin addContainer command has a new option, --resourceSearch path1,path2,... Use this option to designate a resource search path for the newly added container.

 API Changes 

TimeService Updated

The com.streambase.sb.TimeService interface in the Java Client API was updated and extended to work correctly with all time sensitive operators. This interface allows you to construct StreamBase JUnit tests for EventFlow modules that include time sensitive operators, such as a Metronome, or an Aggregate operator with a time-based window. You can also now use a feed simulation to adjust time values in a running test.

For an overview of the TimeService API, see Using the StreamBase JUnit TimeService page in the Test/Debug Guide.

The updated TimeService support provides a new way to specify whether the time values in a JUnit test run of StreamBase Server are controllable, using a new element added to the syntax of StreamBase Server Configuration files, <time-service-configurations>. For example, add:

<time-service-configurations>
  <type>CONTROLLABLE</type>
</time-service-configurations>

The updated TimeService support is accompanied by the following related changes:

  • Three new expression language functions: advanceTimeBy(), advanceTimeTo(), and getTargetTime().

  • Two new jsbadmin commands, getTargetTime and fastForwardTime, described above and on the jsbadmin reference page. (These commands were not added to the sbadmin command.)

  • In previous releases, conditional breakpoint expressions could not call module dependent functions such as getPath() or getNodeName(). This is now allowed.

  • In previous releases, you could not call module dependent functions with the eval() function. This is now allowed.

Specify Description Text for Custom Operators or Adapters

Developers of custom operators or adapters can add description text using the new method setDisplayDescription() in the base Operator class. Description text appears above the OK button in the Load StreamBase Samples dialog.

New Features for Custom Operators

The Java Client API has the following new features for developers of custom operators or adapters:

  • Two Operator new methods, getDynamicVariablesSchema() and getDynamicVariables()allow operators and adapters to access dynamic variables.

  • Custom operators can also now access named schemas with the Operator.getNamedSchema() method.

  • Custom operators can use the new Operator.evaluate() method to evaluate expressions within an operator or adapter's custom code.

  • Custom operators can now retrieve the schema of the Error port with Schema.getErrorSchema().

Autocloseable Support

StreamBaseClient and StreamBaseAdminClient now implement the Java 8 Autocloseable feature.

 Adapter Updates 

New WebSocket Client Read and Write Adapters

The adapter collection now includes WebSocket Client Read and WebSocket Client Write adapters, with a sample in the Samples Guide.

Visual ID for Asynchronous Input Ports

The input ports of certain file-reading adapters is by nature asynchronous, in that file reads are buffered, and one tuple read on the input port does not result in an output tuple every time. To identify such adapters, the input ports of adapters in this category now have a circle around the port's dot and a green-colored background in the circle. Adapters in this category include: Binary File Reader, CSV Reader, External Process Operator, File Reader, FTP Operator, Regex Reader, RSS Reader, and the StreamBase Admin Operator.

File Reader Adapter Updated

The File Reader Input adapter has a new Read Files Synchronously option, for cases when you need files read in the same context instead of in the default queue method.

Log Output Adapter Updated

The Log Output adapter was updated to optionally emit its output in JSON format.

JSON to Tuple Adapter Updated

The JSON to Tuple adapter, introduced in release 7.5.2, was updated to handle incoming JSON documents with nested fields to an arbitrary depth.

HDFS Adapters Updated

The family of HDFS adapters were updated to provide a Buffer Size option for HDFS Read operator, and to provide Buffer Size, Replication Count, and Block Size options to the HDFS Write operator.

 Platform Changes 

As of TIBCO StreamBase release 7.6.0:

JDK and Java Version Changes

On all platforms, the 64-bit Oracle JDK version bundled with StreamBase for private use by StreamBase and Live Datamart applications is now 1.8.0_u51. JDK 8 is now required to run StreamBase Server and StreamBase Studio.

The embedded JDK is used to launch StreamBase Server, StreamBase Studio, LiveView Server, and all StreamBase and Live Datamart applications. As before, you can override this installed default to use a different supported JDK, as described in Using an External JDK.

The Java Client Libraries for StreamBase and LiveView are fully compatible with Java 7, but we recommend using Oracle Java 8, preferably 1.8.0_u51 or later, to compile and run your own client applications, custom function, custom adapters, and monitoring applications. The Java Client Libraries themselves are compiled with target set to Java 1.7 to maintain compatibility.

As before, TIBCO supports and recommends a customer-installed instance of Azul Zing for deployments of TIBCO LiveView servers on Linux x86_64 platforms when very large heap sizes are required. Contact Azul Systems to obtain and license Zing for LiveView. When Zing is installed in its default location, LiveView Server detects and launches with it by default. TIBCO LiveView servers support Azul Zing versions that correspond to Oracle JDK 8 versions.

Deprecated and Removed Features 

The following StreamBase features are deprecated as of release 7.6.0 and will be removed in a future release:

  • Support for building applications with Java 7 using the StreamBase and Live Datamart Java Client Libraries.

  • Support for StreamBase developer machines running 64-bit Windows Vista Business.

  • Support for StreamBase developer and deployment machines running Windows Server 2008.

  • Support for StreamBase deployment machines running Red Hat Enterprise Linux 5.

  • Support for StreamBase deployment machines running Solaris 11 for Intel.

  • Support for converting StreamBase EventFlow modules to StreamSQL modules. (Support for StreamSQL itself is not deprecated; only the conversion is deprecated.)

The following StreamBase features were removed as of release 7.6.0:

  • As discussed above in Updates to Packaging, the TIBCO StreamBase® Adapter for FXall Relationship Trading Customer is no longer provided.

  • Support is removed for building StreamBase client applications for Windows with Microsoft Visual Studio 2005 (Visual C++ 8.0).

  • Support is removed for the sbtest command and running StreamBase Tests in Studio. Use the sbunit command and the StreamBase JUnit subsystem instead. This change has the following considerations:

    • The JAR file sbtest.jar, which was previously deprecated, was removed from the product. Existing StreamBase applications that specify this JAR on their classpath will fail to start until the classpath is corrected. This includes any projects previously created in Studio with the Enable support for running StreamBase Tests option set. Projects that use Studio's automatic configuration management capability can correct this situation by directly editing the sbd-studio.sbconf file, which will cause it to be regenerated.

  • All support for running StreamBase Server or StreamBase Studio on 32-bit Windows is now removed. This change has the following consequences and considerations:

    • There is no longer a need to distinguish between 32-bit and 64-bit StreamBase Command Prompts; thus, the TIBCO Start menu entry StreamBase 7.6 Command Prompt now opens the 64-bit command prompt window.

    • Similarly, the jdk64 folder in STREAMBASE_HOME is renamed jdk, and contains the bundled 64-bit JDK.

    • The Main tab of a StreamBase Application's Run Configuration dialog no longer provides the prompt: Use 32 bit StreamBase Server instead of 64-bit version.

    • Building 32-bit client applications with all supported languages is still available. That is, 32-bit client API libraries for Java, C++, .NET, and Python are still provided.

    • Some StreamBase-provided utilities such as sbc.exe are still built and delivered as 32-bit applications.

    • TIBCO LiveView Desktop is still delivered as a 32-bit application, and can be installed and run on 32-bit Windows machines.

 Documentation Updates 

The documentation for the combined release of TIBCO StreamBase 7.6.0 and TIBCO Live Datamart 2.1.0 included the following updates independent of the above updates:

Wording Change

In previous releases, StreamBase documentation described the opposite of hygienic modules as flexible modules. Starting with release 7.6.0, such modules are described as non-hygienic, which is a clearer term that needs no prior explanation.

Maven Integration Instructions Updated

The instruction page for integrating the m2e Eclipse Maven plug-in with StreamBase Studio was rewritten and corrected.

StreamBase 7.5.x Release History

This sections lists the significant changes in the StreamBase 7.5.x release series.

New in StreamBase 7.5.5

Fixes

TIBCO StreamBase® 7.5.5 fixes several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

Updates to Packaging

Connectivity Package Updates

The TIBCO StreamBase CEP Connectivity Package contains several premium adapters that have separate licensing concerns, and are not included in the base TIBCO StreamBase installation. If you licensed any of these adapters, install the StreamBase Connectivity Package from the TIBCO download site.

The contents of this package are subject to change from release to release. Starting with releases 7.5.5 and 7.6.0, the following adapter is no longer part of either the Connectivity Package or the TIBCO StreamBase base kit:

  • The TIBCO StreamBase® Adapter for FXall Relationship Trading Customer

Updates from Previous Releases

TIBCO StreamBase® 7.5.5 inherits the updates and new features added to maintenance releases of previous release series, through and including release 7.4.4.

StreamBase Platform Updates

Installation Slowness on Mac OS X Resolved

In previous releases, on any Mac with OS X Yosemite or later, double-clicking one of the DMG distribution files that contains StreamBase plus Live Datamart, or LiveView Desktop, could start a spinning beach ball timeout period that could last for many minutes. An alternate installation method was described in the Installation Guide.

Starting with release 7.5.5, TIBCO StreamBase DMG files now have a workaround for the Apple bug that was creating this situation. Thus, this issue is now completely resolved, and no longer requires the alternate installation method.

New and Updated Features

StreamBase release 7.5.5 adds the following updates and new features:

ActiveSpaces Operators Updated

The <adapter-configuration> for the TIBCO ActiveSpaces operators was updated to accept an identity_password value, as specified in the ActiveSpaces documentation. The new configuration element for this purpose is <setting name="identity-pwd" val=...>, and its value can be enciphered with the sbcipher command in the same was as for the authentication-pwd value. The documentation for the operators was updated, and the sbd.sbconf file in the ActiveSpaces Operators sample now illustrates this feature.

A new operator parameter named Query Limit was added to the ActiveSpaces Browse and Listen operators. When set to a positive non-zero value, this feature serves to override the space's query limit (that is, the maximum number of tuples to return when running a query).

Bloomberg B-Pipe Adapter Updated

The Bloomberg B-Pipe input adapter was updated to emit SubscriptionFailure events on its status port.

FX Trading Components Updated

The FX Emulators in the Trading Components Framework were updated to provided initial values for each FX instrument that are a close approximation of the actual live price for each instrument at the time the Emulator runs.

Documentation Wording Change

In previous releases, StreamBase documentation described the opposite of hygienic modules as flexible modules. Starting with releases 7.5.5 and 7.6.0, such modules are described as non-hygienic, which is a clearer term that needs no prior explanation.

Documentation: Maven Integration Instructions Updated

The instruction page for integrating the m2e Eclipse Maven plug-in with StreamBase Studio was rewritten and corrected.

New in StreamBase 7.5.4

Fixes

TIBCO StreamBase® 7.5.4 fixes several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

Updates from Previous Releases

TIBCO StreamBase® 7.5.4 inherits the updates and new features added to maintenance releases of previous release series, through and including release 7.4.4.

New and Updated Features

StreamBase release 7.5.4 adds the following updates and new features:

New JMS and TIBCO EMS Adapters

This release introduces a suite of JMS and TIBCO EMS adapters that replace the JMS and EMS adapters provided in previous releases. The new adapters are implemented as a suite of global Java operators, five for TIBCO EMS busses and five for JMS busses. The new operators store JMS configuration information in the StreamBase Studio project's primary .sbconf file, and no longer in independent .jmsconf files. The JMS Configuration Editor was updated to read from and write to the new .sbconf location, and can import settings from existing .jmsconf files. Features of the new adapters include:

  • Can connect to JMS and EMS resources using SSL.

  • Support for list and tuple data types in JMS-EMS messages.

  • All JMS and EMS operators in an application can share a single connection to the same server.

  • Each JMS or EMS operator can now independently select different configured servers.

  • Support for TIBCO EMS's EXPLICIT_CLIENT_ACKNOWLEDGE mode and Apache ActiveMQ's INDIVIDUAL_ACKNOWLEDGE mode is included.

  • The default message converter can optionally publish null fields.

The replaced JMS and EMS adapters remain part of StreamBase to support existing applications that use them, but are now deprecated. TIBCO strongly recommends migrating any StreamBase modules that connect to JMS or TIBCO EMS message servers to take advantage of the features of the new adapters.

New HDFS Adapters

The basic file reader and writer adapters included with StreamBase now each have a corresponding new adapter that can read from and write to files on HDFS-managed assets. The new adapters have the same feature sets as their corresponding regular file version, but with the addition of a User property setting. HDFS Adapters that have a control port can send a User setting through the control port. There is one exception about same features for corresponding adapters: the HDFS CSV File Reader does not have a tail mode, because that feature is not supported on HDFS. The eight new adapters are:

HDFS Binary Reader HDFS File Reader
HDFS Binary Writer HDFS File Writer
HDFS CSV Reader HDFS Regular Expression Reader
HDFS CSV Writer HDFS XML File Writer
New TIBCO ActiveMatrix BPM Adapter

This release introduces a new adapter to interact with a TIBCO ActiveMatrix BPM server. The new adapter is implemented as a global Java operator, and wraps the ActiveMatrix BPM REST APIs to allow for any of the REST methods to be easily called from within a StreamBase application. The new adapter is described in Using the TIBCO ActiveMatrix BPM Operator and has a sample described in TIBCO ActiveMatrix BPM Operator Sample.

Web Reader Input Adapter Updated

The Web Reader adapter has a new property, Maintain Line Separator. If enabled, this property maintains in the output result the newline and carriage return characters produced by the connected web server.

IBM WebSphere MQ Adapter Updated

The IBM WebSphere MQ Input Adapter now supports the vendor's Read Ahead option. The Output adapter now supports asynchronous writes, when used with IBM MQ versions 7.0.1.9 and 7.1.0.2 or later.

Updates for Custom Feed Simulation Plug-in API

This release includes updates to the API for creating custom file reading plug-ins to be used with the Data File option when defining Feed Simulations. The changes clarify the distinction between custom CSV file reading plug-ins and plug-ins that can read any file type, including binary files. The changes correct several inefficiencies in the previous API when using non-CSV reader plug-ins. The documentation page for this feature was updated, and the sample that illustrates custom plug-ins was reworked for clarity, with its documentation thoroughly rewritten.

EventFlow Debugger Can Now Evaluate Expressions

The EventFlow Debugger gains the ability to evaluate simple expressions in the Eclipse Display view while debugging EventFlow modules. Such expressions can include numeric and string literals, operators, dynamic variables, constants, StreamBase expression language functions, and references to fields in the tuple flowing through the arc with the current breakpoint.

New File Template Sample

This release includes a new sample that illustrates the use of the TIBCO StreamBase® File Reader Adapter and File Writer Adapter working together to read a templated file into the system, then using the inbound tuple to replace values in the read file and then writing that information back out to another file. See File Template Sample.

Platform Changes

As of TIBCO StreamBase release 7.5.4:

  • The list of supported JDBC databases now includes Oracle 12c.

  • The JDK 8 versions supported as an external JDK now ranges from 1.8.0_U40 through U51. The primary JDK shipped with StreamBase is still 1.7.0_U79.

  • The version of the Apache Hadoop libraries included to support the HBase and new HDFS adapters is now 2.7.1.

Documentation Updates

As of this release:

New in StreamBase 7.5.3

Fixes

TIBCO StreamBase® 7.5.3 fixed several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

Updates from Previous Releases

TIBCO StreamBase® 7.5.3 inherited the updates and new features added to maintenance releases of previous release series, through and including release 7.4.4 and release 7.3.12.

New and Updated Features

StreamBase release 7.5.3 added the following updates and new features:

New Apache HBase Adapter

The TIBCO StreamBase® Adapter for Apache HBase is implemented as a suite of five global Java operators, including the HBase Admin, Delete, Get, Put, and Scan operators. See the HBase Operator Samples to learn how these operators can be used.

OSI PI Adapter Enhancements

The OSI PI adapter was extended to support the ability to write to a connected PI database using the PI interface. Connections to both PI and AF servers are now disconnected and reconnected before each batch of requests is sent, to avoid potentially stale connections. And finally, reconnection attempts first try to use the last known good set of connection credentials, falling back to the adapter's Property view credentials as necessary.

Metronome Operator Enhanced

The Metronome Operator was enhanced to accept a Maximum Tuples to Output integer. This allows a Metronome operator to emit a tuple periodically, then stop emitting altogether when the specified limit is reached.

Requirement for AS Clause with tuple() Lifted

In previous releases, you were required to use AS clauses to name fields when using a tuple() expression in both StreamSQL and LiveView's LiveQL. This restriction was lifted such that if you do not provide AS clauses and field names, generic field names are generated for you. For example, the following invocation of tuple(1, 2) generates the field names expr_0 and expr_1:

sbd --eval "tuple(1, 2)"
((expr_0 int, expr_1 int)) 1, 2
sbargen Command Syntax Enhanced

As of this release, the sbargen command has a new -w path option that sets the working directory of the sbargen command. When this parameter is specified, at runtime, sbargen sets its working directory before any other files or paths are read. When -w is used, all files and paths specified elsewhere on the sbargen command line MUST either be absolute paths OR be relative to the directory specified with -w. This includes all input and output files. At runtime, the specified path is checked for existence, and the command halts with an error if the path does not exist.

Schema Compare Dialog Ignores Field Order

The Schema Compare dialog now defaults to ignoring field order.

Web Reader Adapter Sample Updated

The sample for the Web Reader adapter was updated and extended to clarify how to use the adapter in SOAP and RESTful web environments.

TIBCO FTL Adapter Enhanced

The TIBCO StreamBase® Output Adapter for TIBCO FTL® can now be configured to send tuples in batches instead of one at a time, and to specify a buffer flush interval time. This greatly improves the maximum throughput of the adapter.

Expression Language Updates

The StreamBase Expression Language has the following new features as of this release:

  • A new aggregate function emittedByDim() returns a string representing the name of the first dimension that causes the current aggregate window to emit.

  • The expression language supports a new infix operator, <==>. This is a null-safe equals operator that behaves much like the existing == or = operators, except it does not return nulls if any of its operands are null.

  • StreamBase expressions can now express positive and negative infinity with the new positive_infinity() and negative_infinity() functions.

    sbd --eval "positive_infinity()"
    (double) Infinity
    
    sbd --eval "negative_infinity()"
    (double) -Infinity 
    
StreamBase Client API Updates

The SchemaMismatchException class was moved into the Java Client API so that custom adapters have access to it without having to build against the Server runtime.

Platform Changes

As of TIBCO StreamBase release 7.5.3:

  • The Oracle JDK version bundled with StreamBase is now JDK 1.7.0_u79.

  • Support is added for using Oracle JDK 8 update 45 or later as an external JDK (but JDK 1.7 remains the formally supported JDK version).

  • Support is added for running on Red Hat Enterprise Linux 7.x, with the understanding that there are known issues regarding error messages emitted by the StreamBase and Live Datamart RPM-based installer; and in addition, using rpm -e fails to uninstall StreamBase. However, installing by means of the archive installer has no problems and is the recommended installation method for RHEL 7. Once installed on RHEL 7, StreamBase and Live Datamart run as expected, as shown by extensive testing at TIBCO StreamBase.

  • The TIBCO Enterprise Runtime for R operator was validated to support TERR version 3.2.

New in StreamBase 7.5.2

Fixes

TIBCO StreamBase® 7.5.2 fixed several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

Updates from Previous Releases

TIBCO StreamBase® 7.5.2 inherited the updates and new features added to maintenance releases of previous release series, through and including release 7.4.4 and release 7.3.12.

New and Updated Features

StreamBase release 7.5.2 added the following updates and new features:

Import Schema Definitions from XSD Files

The Copy Schema From dialog was extended to accept files in XML Schema format as well as CSV format. This allows you to create StreamBase named schemas by importing schemas saved from other TIBCO products such as TIBCO BusinessEvents, TIBCO BusinessWorks, and others. See Copying Schemas.

Studio: Manual Input View Updates

Release 7.3.2 quietly introduced the JSON input panel, an alternate way to send tuples to a stream in the Manual Input View. The current release extends the usability of the JSON input panel and of the Manual Input View in general with the following new features:

  • The last-entered field values are now preserved after pressing Send Data, which speeds up the entry of a sequence of similar tuples where only a few field values change. This applies to both the forms-based input panel and the JSON input panel.

  • When you toggle between the forms-based panel and the JSON panel by clicking the round JSON symbol , any field values entered are preserved and converted from one form to the other. You can use this feature as a quick way to construct a valid JSON-formatted string for use in a StreamBase unit test.

  • You can now select a number of tuples from the Application Input view, right-click, and select the new Copy Tuples to Manual Input View menu option. For a single tuple selected, this feature automatically copies the tuple's values to either forms-based or JSON input panels. For more than one tuple selected, this feature populates the JSON input view with the values in the selected tuples. See Copying Tuples Automatically into the View for further details.

The JSON input panel and these new features are now documented on the Manual Input of Data page.

Studio: Content Assistance Improvements

The content assistance provided by Studio when editing StreamBase expressions was improved. New features include:

  • Syntax coloring

  • Bracket and parenthesis matching

  • Support for nested fields (within lists and tuples)

  • Context-sensitive support for choosing function arguments (by type)

  • Context-sensitive hover information

  • Auto-sizing edit controls to accommodate longer expressions

Studio: Press Tab to Rename Next Component

When renaming a component on the EventFlow Editor canvas, you can now press Tab from the rename edit field of one component to jump to the next component in the downstream flow and open a rename edit window there.

Studio: New Disable Scroll Sync for Schema Compare Dialog

The Schema Compare dialog now has a button that disables the automatic syncing of the scroll bars of the side-by-side windows.

New System Property to Support Large LiveView Tables

Both StreamBase and Live Datamart applications can fail to compile, reporting an error similar to code attribute has grown beyond 64k limit. This primarily happens when those applications include very large Query Tables with more than one hundred columns. To work around such errors, you can specify the following system property for your application build processes. This property specifies the rewriting of generated code to eliminate the table size issue, then recompiling.

-Dstreambase.codegen.allow-refactoring=true
New System Property to Disable Initial Query Table Load

A new system property, streambase.querytable.load-initial-data, controls whether the initial data load feature of a Query Table's configuration is honored or not. The property's default state is true. Use this property when your application design needs to load initial table values during development, but needs to start with an empty Query Table for production deployment. In this case, set the property =false in the sbconf file for your production deployment configuration.

New JSON to Tuple and Tuple to JSON Converter Operators

The release introduces two new global Java operators, the JSON to Tuple Converter and the Tuple to JSON Converter. These operators are invoked from the Adapters, Java Operators icon in the Palette view.

New MQTT Client Publish-Subscribe Adapters

This release adds a pair of related adapters that provide a way for a StreamBase application to publish to and subscribe from an MQTT broker server. As described on this FAQ page, MQ Telemetry Transport is an international standard for machine-to-machine communication in the Internet of Things world of connected devices. The new adapters are described in MQTT Client Subscribe Adapter and MQTT Client Publish Adapter. The new adapters have a set of samples described in MQTT Client Publish And Subscribe Samples.

New OSIsoft PI Adapters

This release adds a pair of new adapters that connect to a PI database from vendor OSIsoft. The PI database ecosystem provides a way to manage real-time data and events in an enterprise's infrastructure, and is popular with oil and gas companies. Both new adapters require the OSI SDK, release 2012 or later, to be installed alongside StreamBase. Both adapters are input only and do not provide a way to write to an OSIsoft PI database. One new adapter called OSIPI uses OSIsoft's PiPoint mechanism to interact with a PI database, while the second adapter called OSIPIAF uses OSIsoft's Asset Framework mechanism. Both adapters are described on a combined page in the Adapters Guide, OSIsoft PI Adapters and have a simple sample.

JMS Adapter Updated

The JMS Adapter was updated to support the Simple message type. The JMS Configuration File Editor was updated to support Simple messages, and the adapter's documentation was updated to match.

IBM WebSphere MQ Input Adapter Updated

The IBM WebSphere MQ Input adapter was updated to support non-indexed queues. The adapter's Properties view has a new tab, Get Message Options, with two new properties, as described on IBM WebSphere MQ Input Adapter. The adapter also gained an option to allow you to specify the timeout value in milliseconds for an MQ get operation on the queue.

Exegy Adapter Updated

The StreamBase Exegy adapters now rely on version 3.5.0 or later of the JAR file that implements the Exegy Client API, XCAPI.jar. As before, this JAR file is provided as part of your Exegy installation and is not shipped with StreamBase.

Shared Query Table Sample Updated

The Shared Query Table sample was updated to provide a deployment file that starts the sample's three containers, and to also provide an sbbundle file as an illustration of using that feature.

Platform Changes

As of TIBCO StreamBase release 7.5.2:

  • The Oracle JDK version bundled with StreamBase is now JDK 1.7.0_u75.

  • The version of Eclipse that forms the foundation of StreamBase Studio is now 4.4.2.

  • Support is added for using Oracle JDK 8 update 40 or later as an external JDK.

  • Support for IBM DB2 as an external data store location is discontinued.

  • The version of SLF4J included with StreamBase was updated to 1.7.10. The version of Logback was updated to 1.1.2.

  • The standard libraries used by the Thomson Reuters RFA adapters were updated to version 7.6.0.1L1.

Documentation Updates

The StreamBase documentation for release 7.5.2 included the following changes:

New in StreamBase 7.5.1

Fixes

TIBCO StreamBase® 7.5.1 fixed several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

Updates from Previous Releases

TIBCO StreamBase® 7.5.1 inherited the updates and new features added to maintenance releases of previous release series, through and including release 7.4.4 and release 7.3.12.

New and Updated Features

StreamBase 7.5.1 added the following updates and new features:

New TERR Operator

This release adds a new Global Java Operator, the TIBCO Enterprise Runtime for R operator (the TERR operator), which allows StreamBase to use TIBCO's implementation of the R language to analyse and manipulate data. The operator requires an installation of TERR 2.7 or later on the same machine running StreamBase Server. The TERR operator is described on a new page of the Authoring Guide, and has a sample installed from FileLoad StreamBase Sample as usual.

New Non-Strict Option for Output Stream Declared Schemas

When specifying a declared schema for an Output Stream, the Properties View now has a check box labeled Do not fail typecheck for missing fields, which is cleared by default. Select this check box to allow the incoming stream's schema to be a subset of the declared schema. Incoming streams with a superset schema still fail typechecking.

Functions Added to Expression Language

This release adds two new functions and extends one function in the StreamBase Expression Language:

  • gaussianrandom() takes the mean and standard deviation of a Gaussian distribution, and returns a double randomly chosen from that distribution. This can be useful for simulating data from any population that tends to follow a normal distribution.

  • uuid() returns a randomly generated universally unique identifier in the manner of java.util.UUID.

  • random() has a new variant. The existing double random() still returns a pseudo-random decimal fraction in the range [0.0 - 1.0), as before. The new variant int random(int maxrange) takes an integer maximum value and returns a positive integer in the range [0 - maxrange).

Pass-Through Field Option for XML Conversion Operators

The Tuple to XML operator now has a new Pass Through Fields property that lets you specify an input field to convert to XML. The operator then copies the schema and values for all other fields to its output stream. When using this option, you must identify an XML Input Field Name for the tuple data to be converted to XML and can declare an XML Output Field Name to contain the XML document. The XML to Tuple operator also now these properties for use in the opposite conversion direction.

Web Server Adapter New Features

The TIBCO StreamBase® Web Server Request adapter now can parse parameters for the HTTP POST request "application/x-www-form-urlencoded" forms type and add the parsed data to the output. The adapter also now returns an HTTP ONLY field for cookies. Cookie fields were re-ordered to enable their direct streaming into the response side.

Specify Time of Daily Roll for CSV File Writer

When setting a Roll Period of Daily for the TIBCO StreamBase® CSV File Writer adapter to close its output and open a new file, three new int properties (hour, minute, second) on the Rolling and Flushing tab of the Properties view now let you specify the exact time of day to perform the roll. Previously, all daily rolls occurred at midnight.

Parameterized Input for File Reader Adapter

The TIBCO StreamBase® File Reader adapter can now replace tokens in non-binary input files with input tuple subfield values at run time. When the new Replacement Data Field Name property is selected, each token in an Input file is replaced by the contents of a subfield whose name matches the token. Specify tokens in the form of StreamBase parameters (${token-name}).

Tokens can include printf-style format strings enclosed in braces. For example, a double value might be specified in the input file by ${temp_celsius{%6.2f}}. There must be an input tuple subfield named temp_celsius to supply a value for the token, such as 26.4 as temp_celsius. Replacement is supported for all StreamBase data types, including tuples and lists. Tokens cannot be replaced by module parameters, only by properly-named fields defined for the adapter's input schema. For more information, see the Replacement Fields section of the File Reader Input Adapter page in the Adapters Guide.

Documentation Updates

The StreamBase documentation for release 7.5.1 included the following changes:

Standalone Help Viewer Removed

The standalone Help Viewer on Windows and the sbhelp utility on Linux and OS X were removed from the StreamBase distribution. The move to Eclipse 4.4 as a base for StreamBase Studio caused complications that were solved for Studio, but could not be resolved for the Help Viewer. Instead, use the the help system invokable from StreamBase Studio's HelpHelp Contents menu, which provides the identical content as the standalone Help Viewer. You can also see the Eclipse Help format documentation on docs.streambase.com, and the website format documentation locally installed as part of Studio, or on docs.tibco.com. See Using the Help System for details.

Migrating 32-bit C++ Custom Functions

Starting with release 7.5.0, StreamBase supports only 64-bit custom C++ native-code functions on Windows. Accordingly, the Building Custom C++ Functions on Windows section of the Creating Custom C++ Functions page of the API Guide now has suggestions for migrating any 32-bit C++ functions you may have written for older StreamBase releases.

New in StreamBase 7.5.0

Fixes

TIBCO StreamBase® 7.5.0 fixed several product limitations, as described in the Resolved Limitations and Known Issues sections of the Release Notes.

Inherited from Previous Releases

TIBCO StreamBase® 7.5.0 inherited the updates and new features added to maintenance releases of previous release series, through and including releases 7.4.3 and 7.3.12.

Updates to Packaging

StreamBase and LiveView Are Now Integrated

TIBCO Live Datamart 2.0.x is now automatically installed as part of TIBCO StreamBase® 7.5.x. As before, TIBCO Live Datamart is separately licensed. However, you no longer need to download and run a separate LiveView installer to access LiveView features.

  • On Microsoft Windows platforms, TIBCO Live Datamart 2.0.x is installed only as part of the 64-bit edition of StreamBase 7.5.x, and not as part of the 32-bit edition. (The installer for the 32-bit Windows edition of TIBCO StreamBase® 7.5.x works the same as before, described in Installing StreamBase on Windows.)

  • The integrated Live Datamart components installed with StreamBase include TIBCO LiveView server, the Studio extensions to support LiveView projects, all LiveView samples, and fully integrated documentation.

  • As before, LiveView Desktop 2.0.x is still separately licensed and still has an independent installer.

  • As before, LiveView Desktop Workspace Manager 2.0.x is included with your LiveView Desktop license, and has a separate installer available the LiveView Desktop download site.

  • For further details on the New and Updated features of TIBCO Live Datamart, see the separate New and Noteworthy for the Live Datamart product family.

New Default and Recommended Installation Locations
  • To better integrate with other TIBCO product families, and to help customers organize multiple TIBCO products on the same system, the combined release of StreamBase 7.5.x and LiveView 2.0.x has new default and recommended installation directories:

    WindowsC:\TIBCO\sb-cep\n.m
    Linux and Solaris/opt/tibco/sb-cep/n.m
    Mac OS X — No default. The recommended location is /Users/username/Applications/TIBCO StreamBase n.m.x

    where n.m and n.m.x represent the current release numbers.

  • TIBCO StreamBase® LiveView Desktop 2.0's new default and recommended installation locations are:

    WindowsC:\TIBCO\sb-cep\n.m\liveview\desktop
    Linux and Solaris/opt/tibco/sb-cep/n.m/liveview/desktop
    Mac OS X — No default. The recommended location is /Users/username/Applications/TIBCO StreamBase LiveView Desktop n.m.x
  • TIBCO StreamBase® LiveView Desktop Workspace Manager 2.0 also has new default and recommended installation locations, described in Default Installation Directories.

Connectivity Package Updates

The TIBCO StreamBase CEP Connectivity Package contains several premium adapters that have separate licensing concerns, and are not included in the base TIBCO StreamBase installation. If you licensed any of these adapters, install the StreamBase CEP Connectivity Package from the TIBCO download site.

The contents of this package are subject to change from release to release. Starting with release 7.5.0, the following adapter is no longer part of the Connectivity Package and now ships with both TIBCO StreamBase and TIBCO StreamBase Standard Edition:

  • The TIBCO StreamBase® Adapter for Exegy

If you have licensed this adapter as part of your StreamBase configuration, you no longer need to install the Connectivity Package to access it.

New and Updated Features

TIBCO StreamBase® 7.5.0 added the updates and new features shown in the following sections.

 Behavior Changes 
On-Disk Query Table Format Changed

As a consequence of a third-party software update that improves database performance, the on-disk format for disk-based Query Tables was changed for release 7.5.0.

Important

If your StreamBase or LiveView application uses disk-based Query Tables with persisted data, you must read and follow the instructions in Reloading Disk-Based Query Table Data to make sure your data remains accessible.

Stricter Typechecking of Inner Module Schemas

In previous release, if the schema of a stream in a hygienic inner module did not match that of the containing outer module, StreamBase would attempt to rectify the mismatch and proceed without issuing an error. In some situations, however, this could cause ArrayIndexOutOfBounds exceptions. Starting with release 7.5.0, if the schemas do not match, you receive a typecheck error saying so.

Caution

If any of your applications have been relying on this lack of strictness, they may now start to fail typechecking. Reported will be: Source and target of the mismatched streams, the number of fields in each, and a list of schemas. To fix, either change the schemas supplied to and expected from modules to have identical field names and datatypes (order of fields is not important), or mark the inner module as non-hygienic.

Change in Default Time Format for CSV Adapters

The default time format for the CSV File Reader, CSV File Writer, CSV Socket Reader, and Bi-Directional CSV Socket Reader adapters was changed to use 24 hour time instead of 12 hour time. The former default format was:

yyyy-MM-dd hh:mm:ss.SSSZ

The new default format is the following, with the hours field now uppercase:

yyyy-MM-dd HH:mm:ss.SSSZ

This change only affects new instances of these adapters.

Small T Icon for Parameterizable Controls is Now Enabled by Default

The StreamBase Studio authoring preference Hide Control to toggle to/from text fields in Java Operator Properties, introduced in release 7.4.2, is now disabled by default. This enables by default the small T icon () that appears next to controls that can accept a parameter, including check boxes, radio buttons, and drop-down menus in the Properties view of Java adapters and custom Java operators.

 Studio: New Features and Updates
Frameworks Now Integrated into Base StreamBase Package

A Framework is a package of StreamBase modules, interfaces, and schemas that simplifies creating and deploying StreamBase solutions for a specific vertical market. In previous releases, Frameworks were delivered as a separate download with a separate installer that integrated the Framework into StreamBase Studio.

Starting with release 7.5.0, Frameworks now ship as a fully integrated part of the base TIBCO StreamBase® package. As before, you load a Framework into StreamBase Studio much like loading a sample, using FileLoad StreamBase Framework.

Trading Components Framework for Foreign Exchange

The first Framework to be integrated into the base StreamBase package is the TIBCO StreamBase® Trading Components Framework, which simplifies the creation of foreign exchange (FX) trading applications. This Framework provides a set of modules and schemas that include market data handlers for 15 FX venues, and execution handlers for 13 FX venues. This Framework greatly simplifies the configuration of FX handlers by providing uniform interfaces and schemas, and providing appropriate defaults for as many properties as possible. The Trading Components Framework includes documentation integrated into Studio Help, a full set of samples, one for each venue, and a command line administrative tool to manage handlers at runtime.

Test/Debug Perspective Link to Unit Test Wizard

For convenience in the SB Test/Debug perspective, a Create StreamBase Unit Test link was added just left of the Clear button in the Application Output and Application Input view. Like the New StreamBase Unit Test button (), clicking the link starts the StreamBase Unit Test Class wizard with defaults set to include tuples currently in those views as the starting point for your new unit test.

 Studio: Operator Updates 
New ActiveSpaces Operators

An updated TIBCO ActiveSpaces connectivity solution provides a set of operators to connect to a TIBCO ActiveSpaces® Metaspace and Space and to allow your StreamBase application to exchange tuples seamlessly with the Space. The updated TIBCO ActiveSpaces operator consists of seven separate operator actions:

  • TIBCO ActiveSpaces Put, which adds to the Space.

  • TIBCO ActiveSpaces Get/Take, which retrieves from the Space.

  • TIBCO ActiveSpaces Lock/Unlock, which performs locking operations on tuples in the Space.

  • TIBCO ActiveSpaces Browse, which lists some or all tuples in the Space.

  • TIBCO ActiveSpaces Listen, which asynchronously informs of Take, Put, Expire, or Evict events on the Space.

  • TIBCO ActiveSpaces Transaction Operations, which perform Begin, Commit, and Rollback operations on the Space.

  • TIBCO ActiveSpaces Control, which allows your StreamBase application to connect to or disconnect from a Space as well as obtain metric information from it such as the number of tuples contained therein.

Noteworthy enhancements include:

  • All ActiveSpaces operators in an application that reference a given Metaspace share a single connection to it.

  • All ActiveSpaces operators now have an input field named passthrough of type tuple that is passed unchanged to the output.

For more information, see Using the TIBCO ActiveSpaces Operators in the Authoring Guide. The previously-existing ActiveSpaces operator, now called the TIBCO ActiveSpaces Legacy operator, is deprecated as of this release, but is still available.

 StreamBase Runtime Updates 
Referencing Added Fields in Field Grid Expressions

In Map and other operators with field grids, it is now possible to reference a field that you add to an output grid in subsequent output expressions, using the modifier output.fieldname. Previously, doing this caused the typecheck error "the field <added-name> is not available from any stream". Now, added fields are treated much like included fields and declared variables in expressions.

Constants and Named Schemas Now Supported in Conditional Breakpoints

In previous releases, the EventFlow debugger did not support conditional breakpoint expressions that included constants and dynamic variables. You can now use dynamic variables, named schemas, and other module constants in a conditional breakpoint expression.

Overriding Existing Field with a Declared Variable

You can now override an existing input field in an output grid with a declared variable of the same name. Studio warns you about the ambiguity, but uses the declared value as specified. To refer in an expression to the existing field, instead of to a declared variable with the same name, qualify the existing field name with either an input.name or input1.name prefix.

Declare Aggregate Expressions

In the Aggregate Functions tab of an Aggregate operator, you can now specify a local variable name for aggregate expressions as well as simple expressions, using the Declare action in the Action column's drop-down list. The DECLARE keyword in StreamSQL also now accepts aggregate expressions.

 Server Configuration Changes 
Override Constant Definitions at Runtime

The schema for StreamBase Server configuration files has a new element, <constants-overwrite>, which is an optional container for one or more <constant-overwrite> child elements. Use these elements to specify new values at run time for constants defined in an application's modules. The values so specified are initialized at run time. For example, you can replace the value of a constant that specifies the name of a file that an adapter or Java operator uses during development with a file name to be used when the application is deployed.

For details, see <constants-overwrite> in the StreamBase Server Configuration File XML Reference.

In addition, a new Java system property, streambase.sbd.allow-overwrite-constants, with a default value of true, provides control over whether constants are allowed to be overwritten at runtime.

Adapter and Java Operator Settings from Configuration Files

The StreamBase Server configuration file schema has another new element, <adapter-configuration> described, with an example, in the Server Configuration File XML Reference. This element can contain arbitrary child elements and values, as long as they are valid XML. Use this element in conjunction with the new Java class ConfigurationChooserPropertyDescriptor described below to allow developers to provide variable, menu-based configuration options in the Properties view of custom adapters and operators. The property descriptor allows developers to specify an Xpath expression that retrieves the contents of child elements placed in an <adapter-configuration> element, and to build a property choice menu at run time from the retrieved elements.

Use this feature to provide adapter and Java operator property settings that can be configured globally across a project to have sets of specific default values that are easily modified by the end-user. By including appropriate Java code and configuration file elements in a project, developers of a custom adapter or operator can allow end-users to select configured property values from choice menus and, once selected, a property can constrain the choices available for other properties. Run the Configuration File Access Sample to see how to add a property chooser that works this way.

 Expression Language Updates 

The following features were added to the StreamBase expression language in release 7.5.0:

New Functions for Determining Intersections of Two Lists

You can now determine whether two lists have values in common and which items they are:

  • intersect(list, list) returns a list containing the common items.

  • has_intersection(list, list) returns true if common items exist, false otherwise.

The input lists can be of different lengths, but must contain the same data type.

New joinlist() Alias for join()

To enable use of the join() function in StreamSQL or LiveView LiveQL contexts without using escaped identifier syntax, the alias joinlist() was added to the expression language. joinlist() takes the same arguments as join() and behaves identically. This change allows joinlist() to be used in LiveView LiveQL contexts without conflicting with the JOIN keyword, which LiveQL inherits as a reserved word by borrowing its parser from StreamSQL. The joinlist() name is an alias for both simple and aggregate versions of join().

Tuples and Lists Can Have a Trailing Comma

When expressing the values of a tuple or list in a tuple literal or list literal expression, or when using the list() or tuple() casting functions, you can now place one (but only one) trailing comma following the last field in a tuple or last item in a list. StreamBase now accepts but ignores such trailing commas. This is a convenience feature that allows you to leave a comma in place so that quickly added new list members or tuple fields are valid whether or not you remember to prepend the new items with a comma. The following examples are now valid in StreamBase expressions.

list(1,2,3,)
tuple("aardvark" as a, "blogosphere" as b,)
 Command Line Utility Updates 
Administration Utility for Trading Component Handlers

The new sb-trading-components-admin command line utility for the Trading Components Framework lets you identify all instances of handlers running on a network, and provides a way to send commands to one or all handlers using JSON over UDP. The tool lets you interactively ping venues, list current connections and subscriptions, subscribe, and unsubscribe, among other common administrative tasks. For details, see sb-trading-components-admin(1).

Specify Timestamps for sbc and jsbc dequeue Commands

Optional timestamp arguments were added for the sbc deq[ueue] and jsbc deq[ueue] commands, which can take a format string as an optional value:

  • sbc deq [--timestamp[="C-style-formatstring>"]]

    The allowable format strings are based on the ISO C standard strftime API for UNIX platforms and on the Microsoft strftime API for Windows platforms.

  • jsbc deq [--timestamp[="Java-style-formatstring"]]

    The allowable string formats are the same as those accepted by the expression language's format_time() function, which are based on the java.text.SimpleDateFormat class.

If the format string is omitted, the format defaults to "YYYY-MM-DD HH:mm:ss.SSS[+/-]TTTT (such as 2014-07-02 13:50:50.8-0400).

 API Changes 
New Property Descriptor for Accessing Settings from Server Configuration Files

The new Java class ConfigurationChooserPropertyDescriptor extends SBPropertyDescriptor, and enables applications to extract adapter and Java operator property settings from a server configuration file. See the Javadoc for details about the new class, which allows developers to specify an Xpath expression that retrieves the contents of child elements. This class supports the new feature described above that allows developers to provide variable, menu-based configuration options in custom adapters and operators.

Notifications Now Given for Null Arguments to Custom Functions

In previous releases, when given null values for arguments that expect a primitive type, custom functions silently returned nulls. As of this release, you can be notified about null arguments via console warnings. You can optionally set a system property to cause the operator that invoked the function to emit a tuple on its error port. For details, see Method Parameter and Return Types on the Creating Custom Java Functions page of the API Guide.

Programmable Text Field Widths

A new property descriptor (setTextWidth(UIHints.TextWidthHints.SIZE), where SIZE can be SHORT, MEDIUM,or LONG) is a UIHints method that lets programmers specify the widths of text fields, drop-down list items, and text in other Properties view controls. Formerly, text fields had a fixed size.

Updates for Custom Java Aggregate Functions

You can now pass arrays and lists to your custom Java aggregate functions, and they can now accept variable argument lists.

 Adapter Updates 
New File Monitoring Adapter

The new TIBCO StreamBase® File Monitor adapter monitors the file system for changes to directories and files. When files are created, modified, and deleted within the scope of a specified path, the adapter emits tuples describing the event.

New HTTP Web Adapter Pair

It is now simpler to use StreamBase applications to serve and respond to Web pages. The new TIBCO StreamBase® Web Server Request adapter and the TIBCO StreamBase® Web Server Response adapter work together to service HTTP and HTTPS requests. Using an embedded Jetty web server, the adapter set enables StreamBase applications to respond to HTTP and web socket requests from external HTTP clients. For details, see Web Server Request Adapter and Web Server Response Adapter.

New 29West Output Adapter for Acknowledging Messages

The new TIBCO StreamBase® 29West LBM Subscribe ACK Output Adapter enables a StreamBase application to acknowledge incoming LBM messages based on a topic and sequence number by performing an ACK operation when explicit ACK is enabled.

New Name for the 29West Adapters

Because of a change in 29West's ownership, the 29West adapters are now provided as the TIBCO StreamBase® Adapters for Informatica Ultra Messaging:

  • The TIBCO StreamBase® Subscribing Input Adapter for 29West LBM

  • The TIBCO StreamBase® Publishing Output Adapter for 29West LBM

  • The TIBCO StreamBase® 29West LBM Subscribe ACK Output Adapter (newly announced in this release; see the previous entry)

New Property for the Activ Financial Input Adapter

The new property Set undefined numeric values to zero allows the TIBCO StreamBase® Adapter for Activ Financial to disambiguate unchanged data from missing data.

Previously, when the value of a field in an incoming update message changed from defined to undefined, the adapter set that field's value in the resulting tuple to null. But fields missing in the update message are also set to null to indicate that the field's value has not changed since the last update. To enable applications to differentiate between undefined fields and unchanged fields, the new property was added to the adapter to cause it to set undefined numeric fields to 0 instead of null. By default, the property is not selected, which preserves the adapter's legacy behavior.

Explicit ACK for JMS Reader

The TIBCO StreamBase® Adapter for JMS reader now partially supports CLIENT_ACKNOWLEDGE, TIBCO EMS' EXPLICIT_CLIENT_ACKNOWLEDGE and Apache ActiveMQ's INDIVIDUAL_ACKNOWLEDGE. If one of these modes is selected, the adapter will wait until the tuple is fully processed by the application before acknowledging the message. The StreamBase application cannot trigger an explicit acknowledgment of the message.

Failover Option for LiveView Query Adapter

The LiveView Query adapter now optionally supports the new LiveView multiconnection failover client API.

Chooser for File Names Added to File Writer Adapters

Rather than requiring that you type a path for to specify a File Name property, the following adapters now have a file chooser that lets you select a file name from available resources:

  • TIBCO StreamBase® CSV File Writer

  • TIBCO StreamBase® Binary File Writer

  • TIBCO StreamBase® XML File Writer

Option to Disable Status Port in XML-Tuple Converters

The XMLToTuple and TupleToXML Java operators now have an option to disable their status port should that flow of tuples not be of use. By default, the port is enabled, maintaining compatibility with previous releases.

 Sample Updates 

The sample modules and applications included in the StreamBase installation were updated as follows:

New Samples for Trading Components

A new set of 16 sample applications demonstrate various ways to use the Trading Components Framework. Each market data handler and execution handler has its own sample, plus one sample that demonstrates connecting to multiple venues. Eight of the samples are packaged with market data emulators that supply fictitious market data, and thus do not require connections to live data feeds to run. See the StreamBase Trading Components Framework Samples in the Samples Guide.

New Sample for New TIBCO ActiveSpaces Operators

A sample for the new TIBCO ActiveSpaces® operators configures all seven operator actions in one application. For information about the new operators, see Using the TIBCO ActiveSpaces Operators. See TIBCO ActiveSpaces Operator Sample for instructions on running the sample. The same sample contains an application to illustrate the legacy ActiveSpaces operator.

New File Monitoring Adapter Sample

The TIBCO StreamBase® File Monitor adapter sample illustrates using the adapter to watch a user-specified directory. Each time a file is created, modified, or deleted in that directory, the adapter emits a tuple that describes the change.

New Sample for Accessing Properties from Configuration Files

The Configuration File Access Sample illustrates how custom Java operators and adapters can pull property values from a server configuration file, providing greater flexibility to deploy applications in different environments.

New Field Grid Usage Sample

To clarify slight behavior changes in release 7.5.0, the new Field Grids Sample shows how to avoid or resolve conflicts, ambiguities, and typechecking errors related to including and adding fields with name conflicts and related usage of declared local variables.

 Demo Updates 

The following new demos were added to the SB Demos perspective in StreamBase Studio:

Internet — Wikipedia Edits Tracker

Taking as input a stream of recorded events from Wikipedia reporting edits made to actual articles, this application tracks the most-edited articles and the most active authors, and issues alerts when the same article is edited more than once within a given period of time.

Retail — Coupon Generator

Illustrates how retailers can analyze customer preferences to stimulate sales by analyzing streams of order data to issue coupons to customers for similar or related items they purchase. Coupons can be added, deleted, and modified according to changes in inventories and buying behavior.

Sensors — Data Monitor

Shows how manufacturing and process industries can use data streaming from multiple sensors to monitor the health of equipment. The application converts units of measurement for temperatures and pressures, outputs the transformed data, and alerts operators if values exceed prescribed thresholds.

TIBCO Live Datamart — Wikipedia Monitoring

This demo uses the same captured, anonymized data from Wikipedia as the Edits Tracker demo above, but then streams that data into TIBCO Live Datamart data tables. The demo goes on to start an instance of LiveView Desktop, which analyzes the incoming data, displaying the results in two tables and on a US state map, showing the state where each anonymous edit is being made. Instructions are provided to connect the demo to a live Wikipedia feed.

In addition to the new demos, one existing demo might appear to be new:

Thomson Reuters Enterprise Platform for Real-Time (TREP-RT) Threshold Alarm

This demo has long been a part of StreamBase Studio, but may have only appeared in installations with certain configuration options. As part of the merger of TIBCO StreamBase and TIBCO Live Datamart, this demo now appears for all users.

 Platform Changes 
JDK and Java Version Changes

On all platforms, installing TIBCO StreamBase with integrated TIBCO Live Datamart also installs an embedded copy of Oracle JDK, version 1.7.0_u71, for private use by StreamBase products. 64-bit platforms get a 64-bit JDK; on Windows platforms, installing the 32-bit StreamBase Studio overlay also installs a second JDK, the 32-bit version of 1.7.0_u71.

The embedded JDK is used to launch StreamBase Server, StreamBase Studio, LiveView Server, and all StreamBase and LiveView applications. As before, you can override this installed default to use a different supported JDK, as described in Using an External JDK.

The Java Client Libraries for StreamBase and LiveView are fully compatible with Java 7, and we recommend using Oracle Java 7, preferably 1.7.0_u71 or later, to compile and run your own client applications, custom function, custom adapters, and monitoring applications. The Java Client Libraries themselves are compiled with target set to Java 1.6 to maintain compatibility with certain third-party libraries.

TIBCO Liveview Desktop has independent Java requirements, as described in the TIBCO Live Datamart New and Noteworthy.

As before, TIBCO supports and recommends a customer-installed instance of Azul Zing 5.10.x for deployments of TIBCO LiveView servers on Linux x86_64 platforms when large heap sizes are required. Contact Azul Systems to obtain and license Zing for LiveView. When Zing is installed in its default location, LiveView Server detects and launches with it by default.

Studio Now Based on Eclipse 4.4

For release 7.5.0 and later, StreamBase Studio is now based on Eclipse 4.4.1. When installing Studio in the form of plug-ins for your independent copy of Eclipse, the provided plug-ins are compatible with Eclipse 3.7, 3.8, 4.2, 4.3 or 4.4. We strongly recommend starting with the latest Eclipse 4.4.x.

Eclipse 4.4 uses an upgraded format for workspaces. StreamBase Studio automatically upgrades existing workspaces when you first use them following installation. For this reason, we strongly recommend preserving a separate backup of your existing workspaces, in case you need to open them in an earlier Studio release to maintain an existing application.

Solaris 11 Support Replaces Solaris 10

Support for running StreamBase Server on Solaris Intel X86_64 now requires Solaris 11, which replaces the former support for Solaris 10.

Important

To run on Solaris Intel, you must set the environment variable LC_ALL (for example, by setting export LC_ALL=C) in the environment used by StreamBase Server.

Microsoft Windows Support Changes

For development workstations, all TIBCO StreamBase products for Microsoft Windows now require Windows 8.x Professional, Windows 7 Professional, or Windows Vista Business. For Windows Server platforms, StreamBase requires Windows Server 2008 or Windows Server 2012, and their R2 variants. Support for Windows XP and Windows Server 2003 was removed.

The Windows Start menu now lists all installed TIBCO products within a new TIBCO folder. Earlier installed releases of StreamBase software are still listed separately on the Start menu. For more, see The Windows Start Menu.

Microsoft Visual Studio Support Changes

This release adds support for using Microsoft Visual Studio 2012 to build StreamBase client applications using C++, and support for using Visual Studios 2012 or 2013 when using a .NET language.

This release removes support for using Microsoft Visual C++ 7.1 (part of Visual Studio .NET 2003) or Visual C++ 8.0 (Visual Studio 2005) to build either StreamBase clients or custom functions. To build custom functions, you must now use Visual Studio 2010. For more information, see Configuring Visual C++ in the API Guide.

Mac OS X Support Changes

TIBCO StreamBase 7.5, TIBCO® Live Datamart 2.0, and TIBCO LiveView Desktop 2.0 are all supported on Macintosh OS X versions 10.7 (Lion) through 10.10 (Yosemite). This release removes support for running on OS X 10.6 (Snow Leopard). The support for OS X 10.10 (Yosemite) is provisional because of an installation issue.

The installation incompatibility with 10.10 Yosemite results from an Apple bug that prevents opening certain DMG installation archives on Yosemite, including TIBCO StreamBase installers. Once installed on 10.10 Yosemite, StreamBase and LiveView run normally. If you encounter a very slow installation or a hang when trying to install a StreamBase product on OS X Yosemite, use the workaround described in the Special Steps section of Installing StreamBase on OS X, or contact TIBCO StreamBase Support for assistance.

Mac OS X Installation Changes

The OS X installation page now recommends dragging your TIBCO StreamBase product folders from their DMG installers to $HOME/Applications, a folder in your home directory, and not to the system /Applications folder at the root of the disk. This new location avoids the possible permissions problems previously encountered when installing add-on packages such as the Connectivity Package.

Deprecated and Removed Features 

The following StreamBase features are deprecated as of release 7.5.0 and will be removed in a future release:

  • The TIBCO ActiveSpaces operator shipped with release 7.4.x was renamed the TIBCO ActiveSpaces legacy operator, and is now deprecated in favor of a new set of ActiveSpaces operators (each handling a subset of available ActiveSpaces operations) to facilitate ease of use.

  • Support for StreamBase Studio and StreamBase Server on 32-bit Windows platforms is deprecated.

  • Support for using Microsoft Visual C++ 9.0 (Visual Studio 2008) to build StreamBase clients is now deprecated.

The following StreamBase features were removed as of release 7.5.0:

 Documentation Updates 

The documentation for the combined release of TIBCO StreamBase 7.5.0 and TIBCO Live Datamart 2.0.0 included the following updates independent of the above updates:

StreamBase and Live Datamart Documentation is Integrated

The StreamBase and Live Datamart documentation sets were integrated in a way that makes it easy to switch back and forth between them. Two-way cross-references allow links in each set to jump to pages in the other. This linking of both documentation sets occurs in both website and Eclipse Help formats.

New Location for LiveView Command-Line References

Reference pages for the lv-client and lv-server command line utilities are now found with the StreamBase command documentation.

Java and .NET Client Library Documentation Combined

The Javadoc for the StreamBase Client Library now includes pages for the LiveView Client Library as well. Likewise, the .NET documentation describes classes for both libraries. You can still open the Javadoc and .NET documentation from either StreamBase or LiveView starting points, but now the results are the same.

Using the Help System

The page Using the Help System was updated to describe related help features and to display images of the latest versions of documentation in Eclipse and online.

Separate Live Datamart New and Noteworthy

The Live Datamart New and Noteworthy describes further changes to the LiveView documentation.

New and Noteworthy Archives

See the New and Noteworthy Archives for descriptions of the updates and new features added to previous StreamBase releases.