General Migration Notes

This topic assembles migration notes from various releases, helping you to upgrade to the latest StreamBase and Live Datamart releases.

Pre-7.5.0 Installation Locations

For reference when dealing with StreamBase installations older than 7.5.0, the following sections document the default locations for StreamBase assets.

Pre-7.5.0 Windows Installation Locations

In StreamBase releases on Windows prior to 7.5.0, the default installation locations were as follows:

Location on Windows Pre-7.5.0 Path
Installation site for both 32-bit and 64-bit StreamBase
C:\Program Files (x86)\StreamBase Systems\StreamBase.n.m
StreamBase Workspace
C:\Users\username\Documents\StreamBase Studio n.m Workspace
StreamBase Studio Configuration
C:\Users\username\AppData\Roaming\
StreamBase\StreamBase Studio n.m.x
StreamBase Manager Workspace
C:\Users\username\Documents\StreamBase Manager n.m Workspace
StreamBase Manager Configuration
C:\Documents\username\Documents\
StreamBase Manager n.m.x

Pre-7.5.0 OS X Installation Locations

In StreamBase releases on Mac OS X prior to 7.5.0, the default installation locations were as follows:

Location on Macintosh Pre-7.5.0 Path
Installation site
Applications/TIBCO StreamBase CEP n.m.x
StreamBase Workspace
$HOME/streambase-studio-n.m-workspace
StreamBase Studio Configuration
$HOME/.streambase/streambase-studio-n.m.x
StreamBase Manager Workspace
$HOME/streambase-manager-n.m-workspace
StreamBase Manager Configuration
$HOME/.streambase/streambase-manager-n.m.x

Pre-7.5.0 Linux and Solaris Installation Locations

In StreamBase releases on Linux and Solaris prior to 7.5.0, the default installation locations were as follows:

Location on Linux and Solaris Pre-7.5.0 Path
Installation site
/opt/streambase
StreamBase Workspace
$HOME/streambase-studio-n.m-workspace
StreamBase Studio Configuration
$HOME/.streambase/streambase-studio-n.m.x
StreamBase Manager Workspace
$HOME/streambase-manager-n.m-workspace
StreamBase Manager Configuration
$HOME/.streambase/streambase-manager-n.m.x

Back Up Existing StreamBase Workspace and Configuration

If you used a previous release of StreamBase Studio on the target machine, create backup copies of your existing Studio workspace and configuration directories, before using StreamBase Studio in the current release. Save the backup copy with filenames that indicate the version number of the release you are archiving. For example, save SB-workspace-7-6-04.zip.

The default location of the StreamBase Studio workspace directory on OS X for Studio 7.5.0 and later is the following:

/Users/sbuser/Documents/StreamBase Studio n.m Workspace

Using Studio 7.4.x and earlier, the default Studio workspace directory is:

/Users/sbuser/streambase-studio-n.m-workspace

You may have specified a different directory the first time you ran StreamBase Studio.

For 7.5.0 and later, the StreamBase configuration directory is stored in a version-specific directory under $HOME/Library. This folder is hidden in Finder by default; either enable hidden files in Finder Preferences, or navigate to the same folder in a Terminal window:

/Users/sbuser/Library/Application Support/com.streambase.sb.sbstudio/n.m.x/Configuration

For Studio 7.4.x and earlier, the configuration directory is:

/Users/sbuser/.streambase/streambase-studio-n.m.x

In these paths, n.m are the major and minor release numbers of the currently installed StreamBase release, and x is the service pack release number.

Using the Field Grids Sample Prior to StreamBase 7.5

The field-grids sample illustrates how fields specified with the Include and Add actions, and local variables created with the Declare action, can be used in both simple and aggregate expressions in output grids. It also shows how to avoid typechecking errors by properly qualifying field names. Prior to release 7.5.0, fields added using the Add action could not be used in expressions within the same field grid, but that restriction was lifted. See Field Grids Sample for more information regarding this sample.

Iterate Operator Prior to StreamBase 7.4

The Iterate operator to emit one tuple for each element in a list. You specify a field of type list in an incoming stream, and the operator emits one tuple for each element in that field's list.

The Iterate operator acquired its output field grid in StreamBase Studio 7.4. Previously, values for each iteration were placed in a prepended output field, the name of which the Output Element Field Name parameter on the Operation Settings tab specified. This parameter was removed in favor of the flexibility of a field grid, which enables any number of output fields to be defined. When running or editing older EventFlow applications that use Iterate operators, the current version upgrades applications by adding a field with the specified Output Element Field Name to the output field grid using the expression each.element to capture the current element. See for Using the Iterate Operator more information about the operator.

Aggregate Operator Prior to StreamBase 7.4

As explained in Aggregate Operator: Field-Based Dimension Options, with a field-based dimension, a new window is established and evaluated based on the value of a numeric field in the incoming tuple. A tuple containing the results of the aggregation is emitted and typically the window is closed when a tuple arrives whose field value exceeds the specified range of the open window. The arriving tuple that triggered the close is placed in the next window.

With a time-based dimension, a new window is managed and evaluated based on the time a tuple arrives. A window closes and emits when the period specified in the Close and emit after expression elapses, even if no tuple arrives. Prior to StreamBase release 7.4, an input tuple was required to close any time-based window in a Field or Time dimension.

Using Dynamic Variables: Before and After StreamBase 7.4

A dynamic variable is a variable you can declare for a StreamBase module that can thereafter be referenced by name in expressions in operators and adapters in that module. In the declaration, you can link each dynamic variable to an input or output stream in the containing module. In versions of StreamBase Studio prior to 7.4, you had to create an input or output stream before you could define dynamic variables. After StreamBase Studio 7.4, you can define dynamic variables before creating a stream, as explained in Using Dynamic Variables.

Also, while placing a stream on the canvas before creating a dynamic variable is normal practice, as of version 7.4.0 it is not a requirement. Updating streams do not need to be connected to any other component, but can be used on their own to update dynamic variables. In the event you do not want a dynamic variable to be updated by any stream, leave the input stream field empty.

Zing ZDK Requirements for StreamBase 7.3.1 Through 7.3.7

As explained in Supported Configurations, TIBCO supports and recommends the Zing ZDK, version 14 or 15, from Azul Systems for use with large heap and/or low-latency StreamBase and Live Datamart applications. For releases 7.3.1 through 7.3.7, your license for those systems included access to only the Zing versions bundled with StreamBase. If you are maintaining one of these StreamBase releases and plan to upgrade, please contact Azul Systems to obtain a compliant ZDK version.

Running an Application with Default Configuration in StreamBase 7.3.x

As explained in Running Applications in Studio, if one EventFlow application references a second EventFlow application in the same subfolder within a project, to run the application you must add the subfolder to the module search path. One symptom of failing to do this is receiving a "module not found" message from the debugger when attempting to step into a referenced module. This behavior differs from StreamBase Studio 7.3.x and previous releases, in which applications in subfolders that referenced other one could run without adding the subfolder to the module search path.

Using Non-Hygienic Modules Prior to StreamBase 7.2

When using a module marked as non-hygienic, the schema of the input to the Module Reference automatically overrides the schema defined in input streams in the module. Non-hygienic modules were formerly referred to as flexible modules in previous releases. See Hygienic and Non-Hygienic Modules for more information.

All modules and interfaces created in releases before 7.2.0 are non-hygienic, and they retain that setting when imported into release 7.2.0 or later.

Query Operator Comparisons Prior to StreamBase 7.2

When comparing two Query operators or two states of one Query operator in Studio's EventFlow Compare Editor, you may see the following warning at the top of one side of the comparison:

This operator must be upgraded before the information on this tab 
can be correctly displayed.

The Query operator's design changed in major ways at StreamBase release 7.2.0, in the number and names of tabs in its Properties view, and in the internal XML structure. Query operators created in releases before 7.2.0 are silently upgraded to the new format when opened, changed, and saved in the EventFlow Editor for 7.2.0 or later.

This message occurs when you try to compare a Query operator created in or upgraded to the 7.2 XML Query operator structure to an earlier version of the same operator, or to a similar Query operator, that is still in the XML format for a previous release. Because the underlying XML formats are so different, no meaningful comparison can be drawn in graphical form. This is especially true for comparisons of the Fallback tab, which did not exist before release 7.2.0.

Workarounds

There are two workarounds for this condition:

Switch to Text Compare

Use the drop-down control just below the top pane of the Compare Editor to select Text Compare. This switches the attempted but failing graphical comparison to a side-by-side comparison of the underlying EventFlow XML. The two XML formats are different, but you may be able to locate the differences between the two states that are not due to XML format change, but are genuine Property setting differences.

Save a Copy of the Older Version and Let it Upgrade

You may be able to save a copy of the older module of the two versions under comparison. Open, change, and save the copy in Studio 7.2 or later, then compare the two versions again.

Of course, you are not likely to want to change the actual historical version of a module stored in Eclipse local history or in your version control system. That would constitute changing the historical record and losing information. But if a graphical comparison of the two states is important, comparing an upgraded copy of the older version may still let you track down the exact Property setting differences you are seeking.

Using the Lock and Unlock Operators Prior to StreamBase 7.2

The Lock and Unlock operators and the Lock Set data construct are deprecated as of StreamBase 7.2.0 and will be removed in a future release. Do not design new applications that use this feature. Contact StreamBase Technical Support if you need help with replacing this feature in an existing application.

Using Modular Configuration Files Prior to StreamBase 7.2.x

As explained in Using Modular Configuration Files, you can break a complex server configuration file into fragments, and then assemble a top-level configuration file by including smaller fragments into the top-level file. This feature allows for greater reusability of standard or site-specific configuration file settings, and allows you to quickly switch between different configurations for development, testing, and production environments.

Include files are supported only for server configuration files, and not for any other StreamBase file type. For example, you cannot include the contents of a StreamBase deployment file into a server configuration file.

Include file support is provided in two ways:

  • With support for the <sb-include> element that is specific to StreamBase Server configuration files, and works only with such files. The <sb-include> element supports StreamBase smart merge features.

  • With support for the XML Inclusions specification, commonly called XInclude, which is a W3C standard method of including elements from one XML file into another XML file. For recent releases, StreamBase extended the XInclude standard to provide smart merge capabilities.

TIBCO recommends using the <sb-include> element over XIncludes.

You use <sb-include> and XInclude differently in server configuration files, depending on your StreamBase release:

Strict XInclude Rules

Follow strict XInclude rules in the following releases:

  • All of release 7.0

  • Release 7.1 through 7.1.7

  • Release 7.2 through 7.2.2

Smart Merge XInclude Rules

Follow StreamBase-extended smart merge XInclude rules in the following releases:

  • Release 7.1.8 and later 7.1 releases

  • Release 7.2.3 and later 7.2 releases

  • Releases after 7.2

Smart Merge <sb-include> Rules and the sbconfutil Utility

Use the StreamBase-specific <sb-include> element and the sbconfutil utility in the following releases:

  • Release 7.1.9 and later 7.1 releases

  • Release 7.2.5 and later 7.2 releases

  • Releases after 7.2

Error Tuple Schema Changed with StreamBase 7.1.0

The schema of the StreamBase error tuple changed starting with release 7.1.0. The primary difference is that the originalTuple field in previous releases was of data type blob, but is now of type string. This change causes a typecheck error in any module created with an earlier release, when loaded into StreamBase 7.1.0 or later, if the module uses Error Streams or error ports.

StreamBase provides a Java system property and environment variable that directs StreamBase Server to emit error tuples using the schema of previous releases. You can set this property to continue working in 7.1.0 or later with a large application in which several modules emit and manipulate error tuples. When you have migrated all modules to take advantage of the new schema, change this system property or environment variable to specify the new error tuple schema.

You can either set the system property for the JVM that will host StreamBase Server or set the environment variable in the environment in which StreamBase Server will run.

System Property or Environment Variable Value Meaning
streambase.appgen.error-schema-version

STREAMBASE_APPGEN_ERROR_SCHEMA_VERSION

1 Emit error tuples using the pre-7.1 schema, with the originalTuple field emitted as a blob.
2 Default setting in 7.1+. Emit error tuples using the 7.1+ schema, with the originalTuple field emitted as a JSON object string.

Minimizing Module Imports in StreamBase 7.1.x

There are several features in StreamBase Studio that help you detect and eliminate any schemas or constants imported into a module that are not currently being used by that module. Minimizing Module Imports describes these features in detail.

When invoking the Remove Unused Imports wizard using StreamBase 7.1.x, the following menu option is at the top level of the EventFlow menu:

Select to make the module of interest currently active in the EventFlow Editor, then select EventFlowRefactorRemove Unused Imports from Studio's top-level menu.

Map to Sub-fields of Tuple Fields Option Prior to StreamBase 7.0

The Map to sub-fields of tuple fields option appears in two places in StreamBase Studio:

  • In the Data File Options dialog invoked from the Feed Simulation Editor, when specifying options for a CSV data file used as input for a feed simulation. This option only appears in this dialog if the schema of the input stream for this feed simulation includes at least one field of type tuple.

  • In the StreamBase Test editor, when specifying options for a CSV data file used as a data validation file for a unit test.

In StreamBase releases before 7.0, this option was labeled Map to Leaf Fields. For more information about this option, see Map to Sub-Fields Option.

Monitoring and Profiling Application Compatibility Between StreamBase 6.x and 7.x

You can view performance statistics for a running StreamBase application, such as the number of tuples each operator processes per second, the time required to process each tuple, and how much CPU time is used by each thread.

StreamBase monitoring utilities from the 6.x release family cannot connect to and monitor StreamBase Servers from a 7.x release. If you receive an unable to connect or Monitoring is disabled on this server message, try again using a 7.x monitor utility.

StreamBase provides the following ways to monitor StreamBase applications:

Profiling is a subset of monitoring that extracts and analyzes performance statistics about the operators and queues in a running application. Profiling can occur interactively while an application is running, or separately, using a file of collected statistics.

You can also create your own performance monitor client utility using the StreamBase monitoring API, as described in Developing StreamBase Monitor Applications.

StreamBase monitoring utilities from the 6.x release family cannot connect to and monitor StreamBase Servers from a 7.x release. If you receive an unable to connect or Monitoring is disabled on this server message, try again using a 7.x monitor utility.

Logging Compatibility Between StreamBase 6.x and 7.x

The StreamBase Java Client library uses SLF4J, the Simple Logging Facade for Java, for all internal logging. TIBCO encourages customers to use the same logging system when developing your custom embedded Java adapters and Java operators. Logging is described in Using StreamBase Logging.

A change in format of logback.xml files between StreamBase 6.x and 7.x means that Logback configuration files that worked well with a 6.x release will continue to work with StreamBase 7.x, but at the cost of a dozen WARNING messages emitted that urge you to switch to the <encoder> format in your configuration files.

For more complex Logback configuration, there is an online wizard for generating Logback configuration files that makes basic logging setup straightforward. The online wizard generates Logback configuration files using the syntax for older versions of Logback that used the <layout><pattern></pattern></layout> format. This is appropriate for configuration files used with the Logback version shipped with StreamBase 6.6.x. For use with StreamBase 7.0.x, convert those layout patterns to <encoder><pattern></pattern></encoder> as illustrated in the logback.xml file in the logging-logback sample.