StreamBase New and Noteworthy

The TIBCO StreamBase® Adapter for Apache HBase is implemented as a suite of five global Java operators, including the HBase Admin, Delete, Get, Put, and Scan operators. See the HBase Operator Samples to learn how these operators can be used.

The OSI PI adapter was extended to support the ability to write to a connected PI database using the PI interface. Connections to both PI and AF servers are now disconnected and reconnected before each batch of requests is sent, to avoid potentially stale connections. And finally, reconnection attempts first try to use the last known good set of connection credentials, falling back to the adapter's Property view credentials as necessary.

The Metronome Operator was enhanced to accept a Maximum Tuples to Output integer. This allows a Metronome operator to emit a tuple periodically, then stop emitting altogether when the specified limit is reached.

In previous releases, you were required to use AS clauses to name fields when using a tuple() expression in both StreamSQL and LiveView's LiveQL. This restriction was lifted such that if you do not provide AS clauses and field names, generic field names are generated for you. For example, the following invocation of tuple(1, 2) generates the field names expr_0 and expr_1:

sbd --eval "tuple(1, 2)"
((expr_0 int, expr_1 int)) 1, 2

As of this release, the sbargen command has a new -w path option that sets the working directory of the sbargen command. When this parameter is specified, at runtime, sbargen sets its working directory before any other files or paths are read. When -w is used, all files and paths specified elsewhere on the sbargen command line MUST either be absolute paths OR be relative to the directory specified with -w. This includes all input and output files. At runtime, the specified path is checked for existence, and the command halts with an error if the path does not exist.

The Schema Compare dialog now defaults to ignoring field order.

The sample for the Web Reader adapter was updated and extended to clarify how to use the adapter in SOAP and RESTful web environments.

The TIBCO StreamBase® Output Adapter for TIBCO FTL® can now be configured to send tuples in batches instead of one at a time, and to specify a buffer flush interval time. This greatly improves the maximum throughput of the adapter.

The StreamBase Expression Language has the following new features as of this release:

The SchemaMismatchException class was moved into the Java Client API so that custom adapters have access to it without having to build against the Server runtime.

As of TIBCO StreamBase release 7.5.3:

The Copy Schema From dialog was extended to accept files in XML Schema format as well as CSV format. This allows you to create StreamBase named schemas by importing schemas saved from other TIBCO products such as TIBCO BusinessEvents, TIBCO BusinessWorks, and others. See Copying Schemas.

Release 7.3.2 quietly introduced the JSON input panel, an alternate way to send tuples to a stream in the Manual Input View. The current release extends the usability of the JSON input panel and of the Manual Input View in general with the following new features:

The JSON input panel and these new features are now documented on the Manual Input of Data page.

The content assistance provided by Studio when editing StreamBase expressions was improved. New features include:

When renaming a component on the EventFlow Editor canvas, you can now press Tab from the rename edit field of one component to jump to the next component in the downstream flow and open a rename edit window there.

The Schema Compare dialog now has a button that disables the automatic syncing of the scroll bars of the side-by-side windows.

Both StreamBase and Live Datamart applications can fail to compile, reporting an error similar to code attribute has grown beyond 64k limit. This primarily happens when those applications include very large Query Tables with more than one hundred columns. To work around such errors, you can specify the following system property for your application build processes. This property specifies the rewriting of generated code to eliminate the table size issue, then recompiling.

-Dstreambase.codegen.allow-refactoring=true

A new system property, streambase.querytable.load-initial-data, controls whether the initial data load feature of a Query Table's configuration is honored or not. The property's default state is true. Use this property when your application design needs to load initial table values during development, but needs to start with an empty Query Table for production deployment. In this case, set the property =false in the sbconf file for your production deployment configuration.

The release introduces two new global Java operators, the JSON to Tuple Converter and the Tuple to JSON Converter. These operators are invoked from the Adapters, Java Operators icon in the Palette view.

This release adds a pair of related adapters that provide a way for a StreamBase application to publish to and subscribe from an MQTT broker server. As described on this FAQ page, MQ Telemetry Transport is an international standard for machine-to-machine communication in the Internet of Things world of connected devices. The new adapters are described in MQTT Client Subscribe Adapter and MQTT Client Publish Adapter. The new adapters have a set of samples described in MQTT Client Publish And Subscribe Samples.

This release adds a pair of new adapters that connect to a PI database from vendor OSIsoft. The PI database ecosystem provides a way to manage real-time data and events in an enterprise's infrastructure, and is popular with oil and gas companies. Both new adapters require the OSI SDK, release 2012 or later, to be installed alongside StreamBase. Both adapters are input only and do not provide a way to write to an OSIsoft PI database. One new adapter called OSIPI uses OSIsoft's PiPoint mechanism to interact with a PI database, while the second adapter called OSIPIAF uses OSIsoft's Asset Framework mechanism. Both adapters are described on a combined page in the Adapters Guide, OSIsoft PI Adapters and have a simple sample.

The JMS Adapter was updated to support the Simple message type. The JMS Configuration File Editor was updated to support Simple messaages, and the adapter's documentation was updated to match.

The IBM WebSphere MQ Input adapter was updated to support non-indexed queues. The adapter's Properties view has a new tab, Get Message Options, with two new properties, as described on IBM WebSphere MQ Input Adapter. The adapter also gained an option to allow you to specify the timeout value in milliseconds for an MQ get operation on the queue.

The StreamBase Exegy adapters now rely on version 3.5.0 or later of the JAR file that implements the Exegy Client API, XCAPI.jar. As before, this JAR file is provided as part of your Exegy installation and is not shipped with StreamBase.

The Shared Query Table sample was updated to provide a deployment file that starts the sample's three containers, and to also provide an sbbundle file as an illustration of using that feature.

As of TIBCO StreamBase release 7.5.2:

The StreamBase documentation for release 7.5.2 included the following changes:

This release adds a new Global Java Operator, the TIBCO Enterprise Runtime for R operator (the TERR operator), which allows StreamBase to use TIBCO's implementation of the R language to analyse and manipulate data. The operator requires an installation of TERR 2.7 or later on the same machine running StreamBase Server. The TERR operator is described on a new page of the Authoring Guide, and has a sample installed from File → Load StreamBase Sample as usual.

When specifying a declared schema for an Output Stream, the Properties View now has a check box labeled Do not fail typecheck for missing fields, which is cleared by default. Select this check box to allow the incoming stream's schema to be a subset of the declared schema. Incoming streams with a superset schema still fail typechecking.

This release adds two new functions and extends one function in the StreamBase Expression Language:

The Tuple to XML operator now has a new Pass Through Fields property that lets you specify an input field to convert to XML. The operator then copies the schema and values for all other fields to its output stream. When using this option, you must identify an XML Input Field Name for the tuple data to be converted to XML and can declare an XML Output Field Name to contain the XML document. The XML to Tuple operator also now these properties for use in the opposite conversion direction.

The TIBCO StreamBase® Web Server Request adapter now can parse parameters for the HTTP POST request "application/x-www-form-urlencoded" forms type and add the parsed data to the output. The adapter also now returns an HTTP ONLY field for cookies. Cookie fields were re-ordered to enable their direct streaming into the response side.

When setting a Roll Period of Daily for the TIBCO StreamBase® CSV File Writer adapter to close its output and open a new file, three new int properties (hour, minute, second) on the Rolling and Flushing tab of the Properties view now let you specify the exact time of day to perform the roll. Previously, all daily rolls occurred at midnight.

The TIBCO StreamBase® File Reader adapter can now replace tokens in non-binary input files with input tuple subfield values at run time. When the new Replacement Data Field Name property is selected, each token in an Input file is replaced by the contents of a subfield whose name matches the token. Specify tokens in the form of StreamBase parameters (${token-name}).

Tokens can include printf-style format strings enclosed in braces. For example, a double value might be specified in the input file by ${temp_celsius{%6.2f}}. There must be an input tuple subfield named temp_celsius to supply a value for the token, such as 26.4 as temp_celsius. Replacement is supported for all StreamBase data types, including tuples and lists. Tokens cannot be replaced by module parameters, only by properly-named fields defined for the adapter's input schema. For more information, see Replacement Fields in the File Reader Input Adapter page in the Adapters Guide.

The StreamBase documentation for release 7.5.1 included the following changes:

TIBCO Live Datamart 2.0.x is now automatically installed as part of TIBCO StreamBase® 7.5.x. As before, TIBCO Live Datamart is separately licensed. However, you no longer need to download and run a separate LiveView installer to access LiveView features.

The TIBCO StreamBase CEP Connectivity Package contains several premium adapters that have separate licensing concerns, and are not included in the base TIBCO StreamBase installation. If you licensed any of these adapters, install the StreamBase CEP Connectivity Package from the TIBCO download site.

The contents of this package are subject to change from release to release. Starting with release 7.5.0, the following adapter is no longer part of the Connectivity Package and now ships with both TIBCO StreamBase and TIBCO StreamBase Standard Edition:

If you have licensed this adapter as part of your StreamBase configuration, you no longer need to install the Connectivity Package to access it.

When a custom function is called with null values for arguments that expect a primitive type, execution continues, but now a console warning is issued and a tuple is emitted on the error port of the operator that invoked the function. Formerly, when given null values for such arguments, custom functions silently returned nulls.

The TIBCO StreamBase® 29West LBM Subscribe ACK Output Adapter enables a StreamBase application to acknowledge incoming LBM messages based on a topic and sequence number by performing an ACK operation when explicit ACK is enabled.

The TIBCO StreamBase® Adapter for JMS reader now partially supports CLIENT_ACKNOWLEDGE, TIBCO EMS' EXPLICIT_CLIENT_ACKNOWLEDGE and Apache ActiveMQ's INDIVIDUAL_ACKNOWLEDGE. If one of these modes is selected, the adapter will wait until the tuple is fully processed by the application before acknowledging the message. The StreamBase application cannot trigger an explicit acknowledgement of the message.

You can now provide an optional third argument to format_time() that specifies a time zone location (such as "US/Hawaii") or abbreviation ("PST"), as was already possible for the strftime function. For details, see the documentation for format_time.

User-defined aggregate functions can now accept array, list, and variable-length inputs.

The new property Set undefined numeric values to zero allows the TIBCO StreamBase® Adapter for Activ Financial to disambiguate unchanged data from missing data.

Previously when the value of a field in an incoming update message changed from defined to undefined, the adapter set that field's value in the resulting tuple to null. Fields missing in the update message are also set to null to indicate that the field's value has not changed since the last update. To enable applications to differentiate between undefined fields and unchanged fields, the new property has been added to the adapter to cause it to set undefined numeric fields to 0 instead of null. By default, the property is not selected, preserving the adapter's legacy behavior.

The default time format for the CSV File Reader and CSV Socket Reader input adapters was changed to use 24 hour time. Formerly, the default format was:

yyyy-MM-dd hh:mm:ss.SSSZ

The new default format is the following, with the hours field now uppercase:

yyyy-MM-dd HH:mm:ss.SSSZ

The StreamBase documentation for release 7.4.3 includes all the documentation updates reported for release 7.4.2. Specific updates include:

This release includes a new graphical editor for configuring the properties of JMS and EMS adapters that makes it easier to specify and reuse name maps and other configuration elements. The editor can also import settings from TIBCO Businessworks™ applications. The new UI eliminates the need to hand-edit .jmsconf XML files under most circumstances. For more information, see StreamBase JMS Configuration File Editor in the StreamBase Studio Reference Guide.

To more easily reuse Java operators and adapters across application contexts, StreamBase Studio now allows you to enter parameters and expressions for Java Adapter and Operator properties that do not use text boxes. In such components, all check boxes, radio buttons, and drop-down menus now optionally display a small T icon . Clicking the icon converts the control to a text field where you can enter the name of a module parameter in ${param} format or a StreamBase expression in =expr format.

By default, the icons are hidden. To display them, clear a new preference check box, Hide Control to toggle to/from text fields in Java Operator Properties, found under Window → Preferences → StreamBase Studio → Authoring. A keyboard shortcut is also available.

Property views of standard StreamBase operators and data constructs do not have this feature. For further information, see Parameterizing Java Operator and Adapter Properties.

To provide complete access to an sbd process by a remote debugger, a launch configuration can now specify that the server's JVM be suspended when the process is launched. The JVM then resumes once the remote debugger attaches to the process. To enable this, use the new Suspend JVM Initially check box in conjunction with the Allow Remote Java Debugging check box when creating a launch configuration. For more information about how this works, see Editing Launch Configurations.

As of this release, the Pattern operator's polling-interval parameter is no longer supported, and causes an error if it is present. To correct your applications, open your EventFlow module's .sbapp file in a text editor and search for the string polling-interval. If found, remove the entire <param> element that contains that string.

The simple function nan() returns a NaN literal as type double. This is the same value returned from the expression sqrt(-1).

The QueryTopN sample was revised and made easier to use by giving its Query Table a simpler schema and making the table self-loading. The sample lets you specify how many top values to output and formats them both as tuples and as a list of tuples.

Two new Java functions were added to the Java Simple Function sample to demonstrate how to handle variable-length argument lists.

Eclipse users who wish to incorporate StreamBase Studio as an Eclipse plug-in can now obtain the latest Studio Eclipse update site as a downloadable zip archive available from the TIBCO distribution site. For more information, see StreamBase Studio Eclipse Update Site. The StreamBase Studio plug-ins can now be used with Eclipse versions 3.7, 3.8, 4.3, and 4.4.

The TIBCO StreamBase® Adapter for CitiFX Options FIX allows a StreamBase application to connect to and exchange FIX messages with the following Citi Options product. See its documentation here.

An option was added to The TIBCO StreamBase File Reader adapter to read a file as binary data into a single tuple as a blob field.

You can now provide an attachment to the Email Sender as a blob, which can be simpler than providing a local file name. In addition, you can also now specify each attachment's content type and encoding.

The following updates were added to the TIBCO StreamBase® Adapter for Bloomberg B-Pipe:

The TIBCO StreamBase® Adapter for ICAP/EBS was updated to use version 1.6 of the EBS FIX specification.

The time-getting functions get_second() ... get_year() now take an optional second argument, a string literal, for example "UTC" or "Europe/Zurich". When called with one (timestamp) argument, the functions return local time. Providing a recognized time zone ID as the second argument returns the requested unit of time for the specified time zone. Use the Java function TimeZone.getAvailableIDs() to get a list of all time zone IDs. For more information, see Specifying Time Zones.

In addition, the strftime() function has a new optional argument to format timestamps for a specified time zone instead of local time.

Like aggregatelist(), the new aggregatelistnonnull() function creates lists of expressions in the aggregate's window, but excludes nulls, emulating the syntax filternull(aggregatelist(expr)).

The new version() function returns the version of the currently executing StreamBase CEP as a string.

The TIBCO StreamBase® Twitter input and output adapters enable a StreamBase application to interact with the Twitter service using the twitter4j library. The adapters receive, send and retweet tweets and issue other commands, such as searching and filtering.

The new TIBCO StreamBase input and output adapters for KCG Hotspot FX allows a StreamBase application to connect to and exchange FIX messages with KCG Hotspot BookFeed Market Data and Order Entry And Execution services. See the Adapters Guide for more information.

To provide more useful diagnostics, SBUnit test failures with multiple missing/unexpected tuples now throw a ComparisonFailure instead of an AssertionFailedError.

StreamBase kits now install runtime components of JDK 1.7_55 (also called 7u55).

Running the Type Coercion Sample shows how same-named numeric field values with certain compatible types can all appear to be carried by a new field with a type compatible with all original values.

The DebuggingBreakpoint sample illustrates setting unconditional and conditional breakpoints on data arcs when debugging applications in StreamBase Studio.

The StreamBase sample explaining how to use JDBC data sources has been completely revised to be more informative. It now describes using JDBC tables in a trading context, illustrates the use of stored procedures, and provides templates for connecting to several databases. As before, you must configure a database in order to run the sample application.

The StreamBase documentation for release 7.4.1 includes all the documentation updates reported for release 7.3.11.

Like aggregatelist(), the new aggregatelistnonnull() function creates lists of expressions in the aggregate's window, but excludes nulls, emulating the syntax filternull(aggregatelist(expr)).

Eclipse users who wish to incorporate StreamBase Studio as an Eclipse plug-in can now obtain the latest Studio Eclipse update site online, as a downloadable zip archive available from the TIBCO distribution site. For more information, see StreamBase Studio Eclipse Update Site. Installing Studio 7.3.12 as an Eclipse plug-in requires Eclipse versions 3.7 or 3.8.

A reference data output port and schema specification were added to the TIBCO StreamBase® Adapter for Bloomberg B-Pipe to enable it to respond to reference data requests. See the adapter's page in the Adapters Guide for details.

Users of the TIBCO StreamBase® Adapter for Bloomberg B-Pipe can now specify which of five types of authentication to use. See the the adapter's page in the Adapters Guide table for details.

The TIBCO StreamBase 29West LBM Subscribe ACK Output Adapter enables a StreamBase application to acknowledge incoming LBM messages based on a topic and sequence number by performing an ACK operation when explicit ACK is enabled.

An option was added to The TIBCO StreamBase File Reader adapter to read a file as binary data into a single tuple as a a blob field.

The default time format for the CSV File Reader and CSV Socket Reader input adapters was changed to use 24 hour time. Formerly, the default format was:

yyyy-MM-dd hh:mm:ss.SSSZ

The new default format is the following, with the hours field now uppercase:

yyyy-MM-dd HH:mm:ss.SSSZ

The Service Provider Status for the TIBCO StreamBase Adapter for Thomson Reuters Enterprise Platform for Real-Time subscribe has been updated to handle the new Thomson Reuters message structure while remaining compatible with the old structure.

The schema for market data requests in the UBS FIX adapter was updated to match version 1.19 of the UBS specification.

The ICAP/EBS adapter was updated to use version 1.6 of the EBS FIX specification.

The StreamBase JDBC Query sample explaining how to use JDBC data sources was completely revised to be more informative. It now describes using JDBC tables in a trading context, illustrates the use of stored procedures, and provides templates for connecting to several databases. As before, you must configure a database in order to run the sample application.

The StreamBase TopN Query sample was revised and made easier to use by making its Query Table self-loading and with a simpler schema. The sample lets you specify how many top values to output and formats them both as tuples and as a list of tuples.

The StreamBase documentation for release 7.3.12 includes all the documentation updates reported for release 7.2.15. Specific updates include:

Four related properties added to the XML-to-Tuple operator provide greater control over converting XML elements to fields:

See Using the XML to Tuple Operator in the Authoring Guide for more information.

Users of FIX Adapters and Trading Components Handlers now can set the log level of a venue at runtime by issuing setLogLevel commands to a control port. Tuple values for the optional Log input field can dynamically set a log level and also control filtering of incoming and outgoing messages. For details, see Adapter Ports And Schemas in the FIX Adapter documentation.

The TIBCO StreamBase® Adapter for TIBCO Enterprise Message Service™ allows a StreamBase application to connect to TIBCO EMS servers and exchange JMS messages. Note that unlike JMS adapters, which can connect to any number of JMS providers, the EMS adapters only enable connections to TIBCO EMS servers.

The TIBCO StreamBase® Email Sender adapter now supports attachments, identified by a static set of file names, a list of file names provided in a designated input field, or both.

The StreamBase Studio XML editor now provides syntax highlighting and autocompletion when editing JMSReader and JMSWriter adapter .jmsconf configuration files.

The following properties were added to the File Reader Input adapter:

The StreamBase documentation for release 7.3.11 includes all the documentation updates reported for release 7.2.15.

The StreamBase Admin operator now has an optional status port that can be enabled to allow the operator to output status tuples to give the user more information during processing. Status tuples for the operator will be emitted for errors and rejections of input tuples. A rejection status tuple will be emitted if there is a problem validating the input tuple. An error status tuple will be emitted if there is an error during processing. Note that errors will also be emitted from the Error Output port, if it is enabled.

The OSX shortcut to refresh a Studio project's Typecheck Environment has changed to Cmd+R, replacing the previous key binding of Cmd+F5, as Apple uses that to toggle VoiceOver.

Searching for appropriate functions in the StreamBase Studio Function Chooser yields more pertinent results because it now matches your search term with any sub-string of a function name, not just initial sub-strings.

The new parsejson_loose() function works similarly to the existing parsejson() function, but can (within reason) ignore JSON elements that do not conform to the schema definition provided.

StreamBase Server configuration files can now have an optional key-password child XML element under the ssl-authentication XML element. Call the new Java Client API method, StreamBaseURI.getKeyPassphrase(), to return the passphrase of the keystore key (defined by the keypass URI parameter). The existing StreamBaseURI.getKeystorePassphrase() method returns the passphrase of the keystore file (defined by the keystorepass URI parameter). For backward compatibility, the keystore password is used to access the key if the key password (the keystorepass parameter) is not present. In that case, StreamBaseURI.getKeystorePassphrase() returns value of the keypass URI parameter.

For details, see the StreamBase Server Configuration File XML Reference and the Java API Guide documentation.

You can now format timestamps output from the TIBCO StreamBase CSV File Writer. The new Timestamp Format field on the Record Formatting tab specifies the formatting of timestamps when Add Timestamp is set to Prepend or Append. The default format is yyyy-MM-dd hh:mm:ss.SSSZ.

The LDAP Authentication sample demonstrates using Active Directory to limit access to certain StreamBase commands to users who can provide authentication credentials associated with one of the StreamBase roles.

The TIBCO StreamBase® Database Change Data Capture Input Adapter allows a StreamBase application to monitor changes to an Oracle or SQL Server database. It detects when records are inserted, updated, or deleted in a database table and can be customized to report changes to particular table columns.

The new TIBCO StreamBase® File Writer Output Adapter writes tuples or single fields from a stream to a file. A sample application has been added that illustrates its use.

The TIBCO StreamBase® File Reader adapter now supports GZIP-compressed files. In addition, its Control and Status ports now have properties that let them be disabled. They are still enabled by default.

The TIBCO StreamBase® Web Reader input adapter has new functionality and associated properties:

The EBS Live XML input adapter now has an optional status port which, if enabled, emits tuples on connection state transitions. Reconnect logic is included in this update.

The TIBCO StreamBase® Log File Output Adapter has a new option allowing the end user to compress frequent messages. Activating the Compress Frequent Messages property compresses messages within a specified number of seconds (the Max window size property). If the Ignore Timestamp Fields option is enabled, compression will take place without regard to timestamp fields, which can change for each tuple.

In addition, new properties have been added for setting the logger name and for specifying a custom format using StreamBase expression language. For details, see the Log Output Adapter documentation.

The IBM WebSphere MQ Output Adapter now has a Queue When Connection Down property that causes it to queue rather than reject tuples when the connection to the MQ server is down. When the connection is restored, the queued tuples get processed.

The TIBCO StreamBase® Adapter for Thomson Reuters Enterprise Platform for Real-Time subscribe now has a Display Already Subscribed Messages At Debug Level property on its Debug tab that causes already-subscribed messages to be logged at DEBUG rather than INFO level.

The TIBCO StreamBase® StreamBase-to-StreamBase input and output adapter host and port properties now default to localhost and 10000, respectively.

Two optional special fields were added to the TIBCO StreamBase® ActivFeed input adapter: UPDATE_EVENT_TYPE and UPDATE_EVENT_TYPE_NAME, which receive the numeric value and name of the event type of update messages.

The TIBCO StreamBase® RSS Reader now has status and command ports. It can also now insert the RSS Feed link into each item output to allow tracing back to the feed it came from.

Support was removed for connectivity to Sybase RAP native databases.

The StreamBase documentation for release 7.3.10 includes all the documentation updates reported for release 7.2.15.

The CSV file reader adapter now has a Preserve Whitespace property, which, when enabled, preserves leading and trailing white space in string fields. For backward compatibility, the property is disabled by default.

The following pairs of adapters have been added to enable two-way socket communication for various data encodings:

See the Adapters Guide for details on the new adapters.

Support for the following adapters is now deprecated:

The StreamBase documentation for release 7.3.9 includes all the documentation updates reported for release 7.2.15.

The FTP Operator in StreamBase Studio now has equivalent functionality to the SBX FTP operator. The new features are:

When using JMS container connections, in the event the JMS server becomes unavailable the container will retry 10 times to reconnect, waiting 5 seconds between each try. You can override this behavior by setting the following values in sbd.sbconf:

<jms-connections-defaults> 
   <param name="server-num-retries" value="XX" /> 
   <param name="server-reconnect-interval" value="YY" /> 
</jms-connections-defaults> 

XX is the number of retries (0 means no retries, max value is 2147483647) and YY is the number of seconds to wait between retries.

If you want to disable the Connection Timeout property of a TIBCO StreamBase® Output Adapter for TIBCO Spotfire® Automation Services, You can now specify 0 for its value.

The File Reader input adapter has a new property, One Tuple Per Line, which is disabled by default. If you enable it, a tuple is emitted for each line read from the file. When disabled, the entire file contents is emitted in a single tuple.

The following pairs of adapters have been added to enable two-way socket communication for various data encodings:

See the Adapters Guide for details on the new adapters.

Support for the following adapters is now deprecated:

If you use Zing, run the sb-install-zing-system-support command to update to the new version of Zing System Tools (ZST).

Due to limitations in some implementations of the rpm command, the TIBCO StreamBase Zing RPM no longer performs a check of Zing system support during installation, because it fails on certain versions of Red Hat Linux. To be confident about your Zing system configuration after an installation or upgrade, run /opt/streambase/bin/sb-install-zing-system-support check and pay attention to the messages that it issues.

The StreamBase documentation for release 7.3.8 includes all the documentation updates reported for release 7.2.15.

StreamBase 7.3.7 delivers new operators:

StreamBase 7.3.7 delivers several new operators:

StreamBase 7.3.7 delivers new input and output adapters.

LiveView has adapters that simplify using LiveView Server with EventFlow applications.

The product sample demonstrating the adapters is available to customers who have LiveView 1.5.1 or higher installed.

When creating a feed simulation from a CSV file, the Feed Simulation Editor now can match column headers to field names even when they are ordered differently. Fields of type tuple can have subtuple values out of order in the CSV file as long as its column headers use fully qualified names (using dot notation, e.g. Price.Open and Price.Close) that match or closely approximate field names. You can still reassign columns to fields in the Data File Options Dialog if the editor makes matching errors.

When setting up a feed simulation from a CSV file, the option Map data file columns to sub-fields now only appears when the stream's schema includes sub-tuples and a CSV file is selected. By default, the option is enabled if the CSV file contains flattened data and not enabled if it contains hierarchical data. These changes decrease the chances of setting the option incorrectly.

The StreamBase Unit Test Wizard now has improved default settings.

See Creating and Running StreamBase JUnit Tests for more information on StreamBase JUnit tests.

In the .NET Client API, the Timestamp class has been changed to better handle conversions to and from .NET DateTime objects. Specifically:

The Runtime Schema Cast Operator in the Java Operators group, which casts its input schema to its configured target output schema at run time, now has its own icon, as shown here.

Placing a copy of a .sbconf file that you have edited to include customized defaults into your SBHOME/etc/ folder and renaming it to studio-sbd.conf now enables the Configuration File Wizard to copy it into a project instead of generating a skeleton configuration file with standard defaults. For more information, see Cloning Customized Configuration File Default Contents in Studio.

The Oracle JDK installed for private use by StreamBase was updated to version 1.6.0_45. (which originally was incorrectly announced in release 7.3.4 as 1.6.0_43). As before, you can override this installed default to use a different supported JDK as described in Using an External JDK. The supported JDK versions are listed in Supported Configurations.

StreamBase Studio now ships with the default Java Compiler Compliance Level set to 1.6, the minimum level when developing Java code for StreamBase projects. If you install Studio as plug-ins to Eclipse, ensure that Java code developed for StreamBase server target Java 1.6. Users of Java 1.5 must upgrade to 1.6, as StreamBase no longer supports JDK 1.5.

StreamBase Java-based commands such as jsbc, jsbadmin, and sbprofile now have a default maximum heap size of 1 GB when launched on 64 bit systems.

A new system property, streambase.decouple.operator.log-level, enables operator and adapter logging levels to be set independently of the system logging level. Its default is false, which keeps the old behavior whereby operators and adapters use the more verbose setting of the component's or the system log-level property value. When set to true, operator and adapter logging verbosity can be set lower than the system logging level.

The log adapter now has an Emit Logged Tuples property, which is enabled by default. Disabling this property causes the adapter not to have an output port and not to emit logged tuples.

The TIBCO Rendezvous Subscribing Input Adapter now has a Use Unique Queue property, which, if enabled, causes the adapter to use a per-adapter-instance event queue rather than the default Tibrv event queue. Using per-instance queues can lower latency in some situations.

The Use Formatted Timestamps checkbox has been deprecated.

A new Timestamp option has been added as a drop-down list box, with choices Prepend, Append and None (default). Use this control to optionally add a timestamp to the beginning or end of each CSV row.

Adding timestamps enables applications to record data from any input or output stream in a top level application or exposed, in a format suitable for playback. Use the timestamped CSV files for feed simulations to play back streams you recorded in real time.

While an application is running, you can tell sbadmin to capture CSV files from input or output streams by issuing commands such as:

For more information, see CSV File Writer Output Adapter.

As an alternative to specifying fields in a Custom Message Format via a numeric index ({0}, {1}, etc.), you can now use field names ({price}, {symbol}, etc.) directly. Custom Message formatting supports escaped field name identifiers but not expressions.

The StreamBaseFIX configuration file now supports an entry named <dictionary-file> (which is only meaningful when <dictionary> is also specified) to allow users to specify a non-standard name or location for the dictionary file. Previously, StreamBase assumed that custom dictionaries were located in the fixaj-dictionaries folder.

The Trading Components DB-Rapid venue can now output missing depth tiers as zero price items as part of the DepthOfBook message when an item becomes stale. This functionality is disabled by default. Add the parameter OutputMissingTiersAsZeroPriceItems to the instance and set it to true to enable this feature.

A new option on the Adapter Properties tab of the TREP-RT Subscribing adapter, Include AttribInfo In All Updates, adds AttribInfo into all update and refresh messages sent to its subscribers. Receiving this data causes the publishing adapter to include AttribInfo in all messages.

The Tuple Properties tab has two new options:

When looking up fields, the Thomson Reuters Schema Designer now automatically detects fields with fractional pricing, and adds both the double and string fields to the available fields users can select to create a schema. For example, if the BID field indicates a price of 88 3/32, the field list will now contain a BID field of type double and also a BID_as_string field of type string. The Thomson Reuters subscribe adapter now also automatically maps any field in the schema with an "_as_string" suffix to the string representation of that field.

The Exegy library has been updated to version 3.1.11. You must upgrade your installation's Exegy library to this version in order to use this adapter.

A new property, Include Subscribe Pass-Thru Field, has been added. When enabled, the Subscribe stream schema is expected to include a field of any type named PassThru, which is is passed through to the Equities, PriceBook, and Commodities output ports.

A new property, Include Basket Pass-Thru Field, has been added. When enabled, the Basket stream schema is expected to include a field of any type named PassThru, which is is passed through to the BasketUpdates and BasketNav output ports.

For other Exegy input adapter updates, see Fixed in 7.2.15 release notes.

The IBM WebSphere Output Adapter has a new property, Quick Connection-Down Detection. If enabled, the adapter polls the base queue periodically to quickly detect connection-down conditions.

The Wombat Publishing adapter can now send messages using any of the ARRAY types, as well as the VECTOR_MSG, and VECTOR_STRING type. Types are mapped as follows:

As of this release, these FIX engines are no longer supported. Use StreamBaseFIX or QuickFIX/J instead.

The StreamBase documentation for release 7.3.7 includes all the documentation updates reported for release 7.2.15.

Java operators and adapters may now have properties of type string[][]. Only lists of pairs of strings are supported (that is, string[n][2]). Use this property type to allow a user to provide a list of string pairs without requiring unique key values, such as Map property types must have.

The following enhancements were made:

The Morningstar Quotes Feed Input Adapter has new properties that allow you to connect and disconnect from the Morningstar servers without stopping your application.

The StreamBase documentation for release 7.3.6 includes all the documentation updates reported for release 7.2.14.

A new Edit Dimension dialog for the Aggregate operator predicate dimension clarifies options for closing and emitting tuples for the current group and all groups of windows. In addition, all emit and close predicates can now be either simple or aggregate expressions. The new dialog is illustrated and described in the updated page Aggregate Operator: Predicate-Based Dimension Options in the StreamBase Studio Authoring Guide.

Beside pressing the zoom-to-fit icon in the StreamBase Studio editor to show the contents of an entire EventFlow diagram, you can now press Ctrl-0 (zero). On the Mac, press Command-0.

You can now display notes in EventFlow diagrams using any available font. The default is the font used to display text in the editor view. To change the font used for notes, go to Window → Preferences, then select General Appearance → Colors and Font → StreamBase → EventFlow Editor → EventFlow Editor Notes, and then click Edit. The example note displayed below uses the Segoe Script font.

This preference is global, so that all notes in all projects display the selected font until you change it.

The debugging Variables view includes a row for each tuple field. Previous to this release, the text appearing in a row's value cell was a complete representation of the data it contained, except for data in cells containing blobs, which was not displayed at all. Creating textual representations of large sub-tuples and lists could cause debugger performance problems, because the time to calculate strings was proportional to the size of the fields. Now, text for fields of type sub-tuple and list is intentionally abbreviated, and the code to calculate that text executes much faster, both in StreamBase Studio and StreamBase server. Also, cells containing blobs now indicate their content. You can control how that content displays by setting preferences. Go to Window → Preferences → StreamBase Studio → Test/Debug to set the Display blobs as hex or ASCII characters option and specify the Maximum blob characters to display.

You can always view the full values of abbreviated tuples by selecting a row in the Variables view. The Details pane at the bottom of the Variables view displays a textual representation of all of the selected row's data.

Version 7.3.5 ships with a local repository of Apache Maven artifacts for StreamBase. There are artifacts for the StreamBase Client library, the StreamBase Test Support library, and a third one for the embedded server. These artifacts are provided under your StreamBase installation directory, in STREAMBASE_HOME/sdk/maven/repo.

For information on using this repository with your own projects, see StreamBase Maven Artifacts in the API Guide.

In previous releases, the vwap function would return NaN if all volumes were zero or if there were no entries in the window. The function has been updated to return null, matching the behavior of avg.

The Log adapter has two new options:

The SNMP input adapter allows a StreamBase application to act as an SNMP manager for receiving SNMP v1 and v2c traps.

The Syslog input adapter allows a StreamBase application to act as a Syslog collector for receiving syslog messages.

The TREP-RT RMDS subscribe adapter now has a Download Service(s) property that lets users restrict the set of Thomson Reuters services from which data dictionaries are downloaded. In addition, Multiple instances of the adapter within a StreamBase application can now share downloaded and locally-loaded data dictionaries.

The RFA/Java library, rfa.jar, used by TREP adapters has been upgraded to version 7.2.1.

The Regex File Reader adapter now has an optional event port, enabled by a new Start Event Port property. It also has a Map Control Port to Event Port property, which can be used to include control port input tuples in event port tuples. The Start Control Port and Start Event Port properties must be enabled to enable the Map Control Port to Event Port property.

The embedded Web Reader input adapter now supports HTTP POST as well as HTTP GET. Two new examples were added to the Web Reader Samples illustrating how to handle POST and SOAP requests.

The XML to Tuple and Tuple to XML Operator Sample group has been reorganized as five sample applications, four of which have been redesigned:

The two -attribute variants illustrate generating and parsing XML attributes. The two -datatype variants illustrate using fields with a variety of StreamBase data types, including primitive types, lists, and nested tuples. xml2tuple-fixml.sbapp demonstrates how to parse a FIXML message.

The two tuple2xml samples now have two input streams each, one which runs through a map operator to supply default values for any tuple fields containing null, and a second which leaves the user's input unchanged. The three xml2tuple sample applications can now be fed from either new canned XML file read by the file reader adapter or from a user-supplied XML string through an alternate input stream (which is used by the automated tests as well). A note on each sample sbapp canvas describes what the sample does and how the canned data is supplied.

Calling an Embedded StreamBase Server from Java, has been added to the Client Libraries group of StreamBase samples that demonstrates how to run applications in an embedded StreamBase Server inside a JVM. Loading the sample opens a cheatsheet to guide you through it.

The functions getInputTuplesEnqueuedTotal() and getInputTuplesDequeuedTotal() in class OperatorInfo have been deprecated and will be removed in a future release. Their replacement functions, getTotalInputTuplesDequeued() and getTotalInputTuplesEnqueued(), provide the same information but return longs instead of ints. Functions have also been added for getting tuple counts for individual ports.

If you use the bundled Zing JVM from Azul Systems, you must upgrade the Zing system with the sb-install-zing-system-support install command, because the Zing license used with the version requires it. If you do not, you will get a license failure on StreamBase startup.

The Trading Components Framework now provides extra data for swaps and forwards. To support this change, the FX handlers of the Trading Components Framework have an additional schema field, SettlDateFar, for the settlement date for the far leg of swaps.

The StreamBase documentation for release 7.3.5 includes all the documentation updates reported for release 7.2.13, as well as the following:

A new set of functions and built-in language features ease the use of StreamBase blobs to store BSON (binary serializations of JSON-like documents) data. Within BSON blobs, StreamBase ints, longs, doubles, strings, timestamps, and blobs map to their respective BSON scalar types, nulls map to nulls, lists map to arrays, and tuples map to objects. The BSON functions consist of:

Associated language features include:

You can find the BSON specification at http://bsonspec.org.

The Gain GTX FIX adapter allows a StreamBase application to connect to the trading infrastructure provided by GAIN Capital, and to exchange FIX messages with it.

The File Reader input adapter reads a text file and emits a tuple with a string field containing the entire contents of the file. The adapter can be configured to read a file at start-up or in response to the receipt of a tuple on its control input port.

As deploying Microsoft .NET client applications from StreamBase Studio has been greatly simplified, it is no longer necessary to use the StreamBase Redistributable Installer. The simplified deployment process was introduced in StreamBase 7.3.0, and requires a .NET 4.0 or later framework on each client machine you deploy .NET applications to. As of this release, the Redistributable Installer has been eliminated and the .NET client documentation updated accordingly.

For information on .NET application deployment, see Creating StreamBase .NET Clients in the StreamBase documentation.

The Oracle JDK installed for private use by StreamBase was updated to version 1.6.0_45. As before, you can override this installed default to use a different supported JDK as described in Using an External JDK. The supported JDK versions are listed in Supported Configurations.

The StreamBase documentation for release 7.3.4 includeed all the documentation updates reported for release 7.2.11, as well as the following:

StreamBase Studio™ now has a redesigned splash screen and program icon.

The StreamBase documentation for release 7.3.3 includes all the documentation updates reported for release 7.2.11, as well as the following:

This release includes a preview of the .NET operator, which is expected to be added to a future release. StreamBase Studio projects can reference Microsoft .NET assemblies with the .NET operator on the Operators and Adapters palette (keyboard shortcut O.). It is a global Java operator that can load client code in the form of a .NET assembly, instantiate a class within that assembly, pass it incoming tuples, and optionally emit tuples provided by the .NET client code. For more information, see Using the Microsoft .NET Operator in the Authoring Guide. Contact StreamBase Support if you have questions or feedback regarding this feature. Until it is fully released, do not use this feature in production deployments.

In addition to being able to set the speed of a feed simulation with a slider, you can now type a speed factor in the text field to the right of the slider, which is now editable. This lets you simulate at any relative feed rate. The rate you specify can be higher than 25x, which is still the maximum that the slider enables.

The Manual Input view now has a round button at at the upper right of the toolbar for manually enqueuing JSON data. Switch to this mode to enter one or more tuples in JSON syntax. Send the data with the Send Data button or by pressing CTRL+Enter.

You can now control how the Log Output Adapter treats null values within string fields of tuples it logs to the Console. That is, a field of type string that contains a null character can be displayed in the Console as either null or "null". The quoted form is ambiguous, as it could either signify a string field has a null value or is a string containing the characters null. Now, in the Adapter Properties tab, when you select the Display Tuple Field Names check box, a new check box beneath it is enabled. This check box, Quote Null Strings, is by default selected, so as to preserve the adapter's existing behavior. To omit quotes around null strings, turn off the Quote Null Strings option by clearing the check box.

When comparing a timestamp with a string literal, the operators ==, !=, <=, >=, <, and > now attempt to interpret the string literal as a time. If this fails, the comparison fails to typecheck. If the string literal does not contain an explicit time zone, it is assumed to have the time zone set in the operating system of the computer compiling the application.

The tuple-to-XML operator now has an Attribute Fields property that can be used to specify a comma-delimited list of fully-qualified tuple field names that should generate XML attributes rather than sub-elements. For example, if the XML generated by the operator for a given input tuple is:

<Top><s>This value comes from tuple field Top.s of type string</s></Top>

Adding Top.s to the Attribute Fields property would result in the following XML:

<Top s="This value comes from tuple field Top.s of type string"></Top>

Aggregate now has predicates to emit and close all windows. You can use this feature in an application that groups tuples to clean up when a specified condition occurs (such as market close), by emitting and/or closing all the windows.

The following Boolean utility string functions have been added:

When you use sbbundle to create a bundle that includes projects in StreamBase Studio that use Eclipse Classpath Variables, you can identify one or more of them using the new --classpath-variable (-V) option. The option is described in Application Bundling.

The StreamBase documentation for release 7.3.2 includes all the documentation updates reported for release 7.2.11, as well as the following:

The Aggregate operator has two new features for this release. The first is that the input1. prefix is still accepted but no longer required when entering field names in an expression for a predicate-based dimension. The second feature is a new check box in the Edit Dimension dialog for predicate-based dimensions, Evaluate Close and Emit predicates before adding the new tuple to the window. The new check box is described on the Predicate-Based Dimension Options page.

In StreamBase 7.3 and later, when queues begin to grow, each additional buffer is twice the previous buffer size, until a default maximum of 1 GB per buffer is reached. After the limit is reached, each additional buffer requested is capped at the limit size. You can change the limit size by setting the following Java property:

streambase.runtime.stream-queue-max-buffer

See StreamBase Java System Properties for more information on this property.

The output from the sbprofile command now includes the number of tuples delivered for each input/output stream.

The StreamBase expression language includes a new aggregate function, lag(), which returns the next-to-last value for field f in an Aggregate operator's current window or in a Query Table column. If the window or table column has only one row or zero rows, lag() returns null. The function accepts all data types and returns the same type as its argument.

This release increments the Python language support in StreamBase in two ways:

See Creating StreamBase Python Clients for details.

New support was added to com.streambase.sb.operator.parameter.UIHints that allows adapter and operator writers to offer String value proposals to the user when typing into a text field. See the UIHints#setProposals() Javadoc for details.

The Saxo Bank FIX adapter allows a StreamBase application to connect to the trading infrastructure provided by Saxo Bank, and to exchange FIX messages with it.

Several standard adapters now have two new controls on the Adapter Properties tab of their Properties views. There is a new check box, Use Default Charset, which is selected by default (with one exception, see the XML File Writer adapter note in the next paragraph). When selected, the default behavior of the adapter is to use the JVM default character set. When you want to configure an application to use Unicode support, you need to configure the adapter in addition to configuring StreamBase Server and Studio. When the check box is cleared, you must specify a standard Java character set in the new Character Set field.

This applies to the following adapters: CSV File Reader, CSV Socket Reader, CSV File Writer, CSV Socket Writer, Regular Expressions File Reader, Regular Expressions Socket Reader, and the Web Reader. The exception is the XML File Writer adapter, which does have the new controls, but whose default remains the US-ASCII character set.

The CSV Socket Reader Input adapter was updated to provide a server mode where the adapter listens on a specified port for incoming connections. In addition, when acting as a client, the adapter now attempts to reconnect if connections fail, using a specified wait period between reconnection attempts. The adapter also has new properties for specifying how to handle incomplete records, and whether to log incomplete record events.

The sample for the CSV File Reader adapter was simplified for clarity. See CSV File Reader Input Adapter Sample.

The CSV File Reader Input adapter and CSV Socket Reader Input adapter have a new property, Lenient Parsing, whose default value is true.

The FIX Output adapter and all adapters that inherit from the FIX Output adapter were updated with the ability to track acknowledgement of the receipt of sent messages. This feature is described in Requesting Outgoing Message Send Confirmation on the FIX Output Adapter page.

The data dictionary for the Trading Technologies adapter was updated to implement version 7.8.1 of the Trading Technologies FIX specification.

The Thomson Reuters TREP-RT Subscribing Input adapter was updated such that CLOSED events, including access denied errors, are now logged at the INFO level, in addition to being sent to the adapter's event port.

For release 7.3.1, StreamBase Studio is based on Eclipse 3.8.1, which provides some underlying Eclipse bug fixes over Eclipse 3.8.0. When installing Studio in the form of plug-ins for a separate copy of Eclipse, you must start with Eclipse 3.8.1. Earlier Eclipse versions are no longer supported for that purpose.

The Oracle JDK installed for private use by StreamBase was updated to version 1.6.0_37. As before, you can override this installed default to use a different supported JDK as described in Using an External JDK. The supported JDK versions are listed in Supported Configurations.

The StreamBase documentation for release 7.3.1 includes all the documentation updates reported for release 7.2.10, as well as the following: