TIBCO FTL Map Operators

Introduction

The TIBCO Streaming® Operators for TIBCO FTL® map operations allow a Streaming application to perform various operations on FTL maps, including getting and setting messages, iterating through a map's messages and removing messages from a map.

  • TIBCO Streaming® Map Get Operator For TIBCO FTL, which allows a Streaming application to retrieve a message with a specific key from a specified FTL map.

  • TIBCO Streaming® Map Iterate Operator For TIBCO FTL, which allows a Streaming application to add a message with a specific key to a specified FTL map.

  • TIBCO Streaming® Map Remove Operator For TIBCO FTL, which allows a Streaming application to retrieve all messages from a specified FTL map in one operation.

  • TIBCO Streaming® Map Set Operator For TIBCO FTL, which allows a Streaming application to remove a message with a specific key from a specified FTL map.

These operators use a connection to the FTL realm server provided by the FTL Control Operator instance that is configured in their FTL Control Operator Name property. Several FTL operators can thus be made to share a realm connection by using the same value for that property.

The operators support a subset of Streaming data types that contains tuples and lists, though lists may not contain elements of type list.

The operators are configured through several properties set in their Properties view in StreamBase Studio.

TIBCO Middleware Dependencies

This operator requires access to the JAR file that implements the TIBCO FTL Java API on your system, and any files referenced by that JAR file. The FTL adapter was tested with FTL version 6.0, and is compatible with higher versions as well. To make the TIBCO FTL Java API available to the adapter, you must supply a maven dependency for it -- specifically, to FTL's ${FTL_HOME}/lib/tibftl.jar.

The tibftl.jar file, in turn, relies on various native libraries such as tibftljni.dll on Windows and tibftljni.so on Linux. As with tibftl.jar, these native libraries are supplied as part of your TIBCO FTL installation and are not shipped with TIBCO Streaming. Make sure to edit your src/main/configurations/engine.conf and make changes to reflect your current setup, to include an externalNativeLibraryPath directive that points to your FTL installation's ${FTL_HOME}/bin directory. Please see the FTL sample application for an example configuration.

Operator Properties

All four operators have the following operator properties:

Property Description
FTL Control Operator Name The name of the FTL Control operator instance that will be managing the connection to the FTL realm server to be used by this operator. If this Control instance is not located in the current fragment, its fully-qualified name should be used. All FTL operator instances using a given instance of the Control operator to manage their realm connection will share this connection.
Endpoint Name The FTL endpoint, which abstracts the transport details.
Remove Map On Disconnect When checked, this causes the operator to remove the map from the FTL realm before disconnecting from it.
Log Level Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.
FTL Library Log Level The log level used by the FTL library. Multiple FTL input and output operator instances share the most verbose level configured in any operator.

Some of the operators also offer additional properties:

MapGet, MapIterate

Property Description
Edit Schema Tab These operators have an additional tab named "Edit Schema", where the expected schema of incoming FTL messages must be specified.

MapSet

Property Description
Message Formats Tab Use this tab to specify the names of the message formats used by the operator. Top-level Message Format describes the name of the FTL message, and the Nested Message Formats list specifies the format names of any nested messages (that is, fields of type Message).

Input Ports

Each of the four FTL Map operators have at least one input port, with schemas described in this section.

MapGet Operator

This operator only has one input port, which receives requests to get the specified entry from a map. The port has the following schema:

Field Name Field Type Description
FTLMapName string Name of the FTL map from which to retrieve the message. If the map doesn't exist, it will be automatically created before executing the operation.
FTLKeyName string Key of the message to fetch in the FTL map.

MapIterate Operator

This operator only has one input port, which receives requests to iterate through the entries in the specified map. The port has the following schema:

Field Name Field Type Description
FTLMapName string Name of the FTL map from which to retrieve the messages. If the map doesn't exist, it will be automatically created before executing the operation.
FTLLockMap boolean If set to true, the FTL map will be locked during the operation to avoid contention from other operators.

MapRemove Operator

This operator only has one input port, which receives requests to remove entries from the specified map. The port has the following schema:

Field Name Field Type Description
FTLMapName string Name of the FTL map from which to remove the message. If the map doesn't exist, it will be automatically created before executing the operation.
FTLKeyName string Key of the message to remove from the FTL map.
FTLLockMap string If set to true, the FTL map will be locked during the operation to avoid contention from other operators.

MapSet Operator

This operator only has one input port, which receives requests to set entries in the specified map. The port expects its schema to have at least the following fields:

Field Name Field Type Description
FTLMapName string Name of the FTL map on which to set the message. If the map doesn't exist, it will be automatically created before executing the operation.
FTLKeyName string Key to assign to the message in the FTL map.

All other fields will be treated as values to be set on the outgoing FTL message.

Output Ports

Each of the four FTL Map operators have at least one output port, to issue status, information and error tuples. The schema for this port is the same for all four operators and is as follows:

Field Name Field Type Description
Status string Describes the type of event (Error, Success, etc.)
Time timestamp The time at which this status tuple was emitted.
Info list<tuple> A list of additional information strings, each contained in a tuple with a name / value pair.

In addition to this status output port, some operators also have other output ports as described in the following subsections.

MapGet Operator

This operator has a second output port to emit incoming FTL messages as tuples.

When it executes a Get operation, the MapGet operator emits a tuple on its second output port representing the FTL message it retrieved from the map. The port's schema is derived from the Edit Schema tab of the operator's property view and can contain a subset of Streaming data types, including lists and (nested) tuples to accommodate hierarchy in FTL messages. FTL message fields are mapped to tuple fields by case-sensitive name. List fields may contain elements of type double, long, boolean (which will be mapped from the corresponding FTL LONG field as FALSE for a value of 0 and TRUE otherwise), string, timestamp or tuple. In addition to the fields specified on the Edit Schemas tab, each tuple will also have two extra fields set automatically to indicate the map it came from and the key it was assigned in the map:

Field Name Field Type Description
FTLMapName string Name of the FTL map from which this tuple was retrieved. If no message was found in the FTL map with the FTLKeyName specified in the Get command, this field will be set to null.
FTLKeyName string Key of the message in the FTL map. If no message was found in the FTL map with the FTLKeyName specified in the Get command, this field will be set to null.

MapIterate Operator

This operator has a second output port to emit incoming FTL messages as tuples.

When it receives an Iterate request, the MapIterate operator emits a tuple on its second output port for each FTL message contained in the specified map, ending with a marker tuple that has null values in its FTLMapName and FTLKeyName fields to indicate the end of the iteration process.

The port's schema is derived from the Edit Schema tab of the operator's property view in a manner similar to the MapGet operator, using identical data type mapping rules. In addition to the fields specified on the Edit Schemas tab, each tuple will also have two extra fields set automatically to indicate the map it came from and the key it was assigned in the map:

Field Name Field Type Description
FTLMapName string Name of the FTL map from which this tuple was retrieved, or null if this is the end of the iteration operation.
FTLKeyName string Key of the message in the FTL map, or null if this is the end of the iteration operation.

Data Type Conversion

In processing incoming FTL messages, the operators convert FTL data types to Streaming data types (or vice-versa). As shown in the table below, each FTL type has a single corresponding Streaming type. When an attempt to perform an invalid conversion occurs, a one-time warning for the field is displayed and the corresponding field in the tuple is set to null (or in the case of Streaming-to-FTL conversions, the FTL field is omitted from the message).

FTL Type Streaming Type
DATETIME timestamp
DATETIME_ARRAY list<timestamp>
DOUBLE double
DOUBLE_ARRAY list<double>
LONG long

Note

For convenience, Streaming fields of type booleans are also converted from FTL LONGs since FTL has no support for boolean data. An incoming value of 0 will set the boolean to FALSE, while any non-zero value will be converted to TRUE. The opposite conversion will be performed when sending messages to FTL.

LONG_ARRAY list<long>

Note

Similarly, Streaming list<boolean> fields are converted from FTL LONG_ARRAYs using the rule described above. The opposite conversion will be performed when sending messages to FTL.

MESSAGE tuple
MESSAGE_ARRAY list<tuple>
OPAQUE blob
STRING string
STRING_ARRAY list<string>

Typechecking and Error Handling

The TIBCO FTL Map operators uses typecheck messages to help you configure the operator in your Streaming application. In particular, the operator generates typecheck messages when a required property has not been specified, the control input schema is missing a required field, or an input or output schema contains unsupported Streaming data types.

The operator generates messages on the status port during runtime under various conditions, including failed FTL operations.

Suspend and Resume Behavior

When suspended, the TIBCO FTL Map Operators stop processing incoming FTL messages and outgoing map requests.

When resumed, the operators resume processing incoming FTL messages and outgoing map requests.