TBF Input Adapter

Introduction

The TIBCO StreamBase® Adapter for Toronto Broadcast Feed (TBF) allows a StreamBase application to receive Securities Trading and Message Access Protocol (STAMP) market data messages from the Toronto Broadcast Feed (TBF) and TSXV Broadcast Feed (CBF). Each adapter instance can be configured for an equity feed, a reconciliation feed, or both. Each feed consists of two sessions: a feed session on which market data messages are received, and a recovery session used to request missed messages from the the STAMP server.

The adapter is embedded in the StreamBase application and has one input port and two or three output ports, depending on its configuration. The input port is used to send commands to the adapter from an input stream specifying dynamic changes in subscriptions or sequence numbers. The first output port emits status tuples to convey important adapter events, such as session and connection changes and errors processing feed messages. The remaining output ports emit tuples when market data messages are received on the equity and/or reconciliation feeds.

The adapter is configured through several properties in the adapter's Properties view in StreamBase Studio. These include the name of a resource file that configures the sessions for communicating with the STAMP server, a symbols file for subscribing to specific symbols at startup, check boxes that determine which of the two feeds the adapter reads, and schema editors for configuring the schemas of the equity and/or reconciliation output ports.

TBF Adapter Properties

This section describes the properties you can set for this adapter, using the various tabs of the Properties view in StreamBase Studio.

In the tables in this section, the Property column shows each property name as found in several properties tabs of the Properties view for this adapter.

General Tab

Name: Use this field to specify or change the component's name, which must be unique in the application. The name must contain only alphabetic characters, numbers, and underscores, and no hyphens or other special characters. The first character must be alphabetic or an underscore.

Adapter: A read-only field that shows the formal name of the adapter.

Class: A field that shows the fully qualified class name that implements the functionality of this adapter. Use this class name when loading the adapter in StreamSQL programs with the APPLY JAVA statement. You can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start with application: If this field is set to Yes or to a module parameter that evaluates to true, an instance of this adapter starts as part of the containing StreamBase Server. If this field is set to No or to a module parameter that evaluates to false, the adapter is loaded with the server, but does not start until you send an sbadmin resume command, or until you start the component with StreamBase Manager. With this option set to No or false, the adapter does not start even if the application as a whole is suspended and later resumed. The recommended setting is selected by default.

Enable Error Output Port: Select this check box to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports and Error Streams to learn about Error Ports.

Description: Optionally enter text to briefly describe the component's purpose and function. In the EventFlow canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Adapter Properties Tab

Property	Description
Configuration File	The name of the file that configures the sessions for communicating with the STAMP server.
Symbols File	The name of an optional symbols file for subscribing to specific symbols at startup. If no symbols file is specified, or the symbols file is empty, the adapter subscribes to all symbols.
Filter Encrypted	If enabled, the adapter discards messages that cannot be decrypted.
Wait For Input	If enabled, the adapter does not start receiving messages until input has been received on its control stream (input port).
Log Level	Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE, and ALL.

Output Ports Tab

Property	Description
Receive Equity Feed	If enabled, the adapter enables an output port that emits a tuple for each equity feed message received. When this property is enabled, the corresponding schema editor in the Edit Schemas tab is enabled as well.
Receive Equity Status/Reconciliation Feed	If enabled, the adapter enables an output port that emits a tuple for each equity status or reconciliation feed message received. When this property is enabled, the corresponding schema editor in the Edit Schemas tab is enabled as well.

Edit Schema Tab

This tab shows you the fixed schema of the status output port. If the Equity Feed or Equity Status/Reconciliation Feed output ports are enabled, this tab provides a standard schema editor with which to specify the schemas of those ports.

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Using the Adapter in a StreamBase Application

This section discusses how to use the TBF Input Adapter in a StreamBase application. As shown in the diagram below (depicting the adapter's sample application), the TBF adapter has one input port and up to three output ports to communicate with the surrounding application.

The TBF Input Adapter's ports are used as follows:

Input port, Command: This input port receives tuples used to subscribe to, or unsubscribe from, symbols after the adapter has started and to change the incoming or outgoing sequence numbers. The Subscribe port has the following schema:
- command, string: Contains the name of the command to execute.
  - subscribe — The adapter subscribes to the symbol specified in the symbol field.
  - unsubscribe — The adapter unsubscribes to the symbol specified in the symbol field.
  - setSenderSequenceNumber — The adapter sets the outgoing (sender) sequence number for the session specified in the session field to the value specified in the sequenceNumber field.
  - setTargetSequenceNumber — The adapter sets the expected incoming (target) sequence number for the session specified in the session field to the value specified in the sequenceNumber field.
- session, string: Required only if the adapter is configured to connect to more than one feed session. Should contain a text value indicating the session type that the command is intended for, such as EquityFeed or ReconFeed.
- symbol, string: Required only if the command is subscribe or unsubscribe. Specifies the symbol to subscribe or unsubscribe from.
- sequenceNumber, int: Required only if the command is setSenderSequenceNumber or setTargetSequenceNumber. Specifies the number that the sequence number for the sender or target should be set to.
Output port 1, Status: This output port emits status, information, and error tuples. The Status port has the following schema:
- type, string: Contains one of the following values describing the type of event that occurred:
  - Session
  - Connection
  - Login
  - Feed Message
  - Process Tuple
  - Suspend/Resume
- object, string: the name of the object associated with the event, such as the session created or logged in to, the contents of the feed message that wasn't processed successfully, or the name of the adapter that was suspended or resumed.
- action, string: the action associated with the event, such as a login succeeding or adapter suspended/resumed.
- message, string: A human-readable message string
Equity Feed Output port: If enabled, the adapter emits tuple on this port each time an equity feed message is received. The port's schema is derived from the Edit Schema tab of the adapter's Properties view. Equity feed message fields are mapped to tuple fields by case-sensitive name. Equity feed message fields not present in the schema of this port are discarded. Fields present in the output port schema but not in the equity feed schema are sent as null in the tuple.
Equity Status Feed/Reconciliation Feed Output port: If enabled, the adapter emits tuple on this port each time an equity status or reconciliation feed message is received. The port's schema is derived from the Edit Schema tab of the adapter's Properties view. Equity status or reconciliation feed message fields are mapped to tuple fields by case-sensitive name. Equity status or reconciliation feed message fields not present in the schema of this port are discarded. Fields present in the output port schema but not in the Equity status or reconciliation feed schema are sent as null in the tuple.

Add an instance of the adapter to a new StreamBase application as follows:

In StreamBase Studio, create a project, and create an EventFlow application file to host the adapter.
Drag an instance of the TBF Input Adapter from the Operators and Adapter drawer in the Palette view to the canvas.
Connect an input stream to the adapter's input port and configure the stream's schema with fields named command, session, symbol, and sequenceNumber of types string, string, string, and int, respectively.
Configure the schema of the equity and/or reconciliation output ports using the Edit Schema tab of the adapter's Properties view to match the set of fields expected in the corresponding feed messages.
Connect output streams to the adapter's two output ports.
Specify the name of a STAMP session configuration file and populate the file with the properties of the equity and/or reconciliation feed sessions and corresponding recovery sessions.

STAMP Session Configuration

STAMP sessions are configured in a settings file consisting of one or more sections, one for each session. Each section begins with a [session] heading, indicating the settings for a new session follow. Within a section, lines of the form Name=Value define the settings for the particular session. A section with a heading of [default] provides default values for settings not explicitly specified for a particular session.

The following table lists the set of STAMP sessions along with the valid and default values for each.

Name	Description	Valid Values	Default
Session
SourceAddress	The local STAMP address associated with this session.	case-sensitive alpha-numeric string	—
DestAddress	The remote STAMP address associated with this session.	case-sensitive alpha-numeric string	—
ConnectionType	Defines whether session is to act as an acceptor or an initiator.	initiator, acceptor	All Feed Sessions are initiators. It is not necessary to specify this parameter for a Feed Session, however the Recovery Session(s) associated with the Feed Session(s) must be specified as `initiator`
TimeZone	Time zone for this session; if specified, the session start and end will be converted from this zone to UTC.	Time zone ID, such as `EST`	—
StartTime	Time of day that this STAMP session becomes activated.	time in the format of HH:MM:SS [timezone]. The time zone is optional. The TimeZone setting will be used, if set, or UTC will be used by default. The timezone string should be one that the Java TimeZone class can resolve. For example, `15:00:00 US/Central`.	—
EndTime	Time of day that this STAMP session becomes deactivated.	time in the format of HH:MM:SS [timezone]. The time zone is optional. The TimeZone setting will be used, if set, or UTC will be used by default. The timezone string should be one that the Java TimeZone class can resolve. For example, `15:00:00 US/Central`.	—
Version	The Version of STAMP being used.	The word "Version", followed by a number (such as Version 5.0)	Version 5.0
SessionType	The type of the session to create.	EquityFeed, EquityStatusFeed, ReconFeed, or Recovery	—
UseMD5SessionAuth	Use the MD5 algorithm to authenticate logging in to the session.	Y/N	Y. Note that this parameter does not apply to Feed Sessions and is only necessary to specify for Recovery sessions.
UseMD5MessageAuth	Use the MD5 algorithm to authenticate each message received from a counterparty.	Y/N	Y. Note that this parameter does not apply to Feed Sessions and is only necessary to specify for Recovery sessions.
SharedPassword	The secret key shared between both parties in a session for message authentication.	any alphanumeric string	Note that this parameter does not apply to Feed Sessions and is only necessary to specify for Recovery sessions.
OESourceAddress	The source address of the Feed session associated with this Recovery session. Used only for Recovery sessions.	any valid STAMP address	—
OEDestAddress	The Destination address of the Feed session associated with this Recovery session. Used only for Recovery sessions.	any valid STAMP address	—
RetransSourceAddress	The source address of the Recovery session associated with this Feed session.	any valid STAMP address	—

Validation
UseDataDictionary	Tell session whether or not to expect a data dictionary.	Y/N	Y
DataDictionary	XML definition file for validating incoming STAMP messages. If no DataDictionary is supplied, only basic message validation is done.	Valid XML data dictionary file. If DataDictionary is not specified and UseDataDictionary=Y, then the server will look for a default dictionary based on the session's VersionNumber (such as Version 4.0 = Version40.xml). The DataDictionary file search strategy is to use a URL, then the file system, then the thread context classloader (if any), and then the DataDictionary instance's classloader.	—

Initiator
ReconnectInterval	Time between reconnection attempts in seconds.	positive integer	30
HeartBtInt	Heartbeat interval in seconds.	positive integer	—
RequestRetransmitOnHeartbeatTimeout	For equity and reconciliation feeds, determines whether a retransmit request is sent after a heartbeat timeout.	Y/N	N
SocketConnectPort	Socket port for connecting to a session.	positive integer	—
SocketConnectHost	Alternate socket hosts for connecting to a session for failover, where n is a positive integer. That is, `SocketConnectHost1, SocketConnectHost2...` must be consecutive and have a matching SocketConnectPort[n].	Valid IP address in the format of x.x.x.x or a domain name.	—
SocketConnectHost<n>	Alternate socket hosts for connecting to a session for failover, where n is a positive integer. That is, `SocketConnectHost1, SocketConnectHost2...` must be consecutive and have a matching SocketConnectPort[n].	Valid IP address in the format of x.x.x.x or a domain name.	—
SocketConnectProtocol	Include the line `SocketConnectProtocol=Multicast` to enable multicast operation for a session. Do not include this line for point-to-point sessions. When using a multicast session, the `SocketConnectPort` and `SocketConnectHost` parameters should contain a valid multicast port and host, respectively.	Multicast	—

Adapter-Specific Settings
logtype	The log type for this adapter to use.	Currently supported types are screen and file.	screen
storetype	The type of storage for this adapter to use for message and sequence number persistence.	Currently supported types are memory (no persistence) and file.	file
PrivateKeyIdentifier	The PrivateKeyIdentifier the adapter will use to determine whether or not the private content portion of the current message is intended for this user.	an integer value as provided by the TSX.	—
DecryptionKey	The DES key for this adapter to use for message decryption.	—	—
AllowTimeouts	Specifies whether this session will time out and disconnect if it does not receive any messages within the time frame specified by the HeartBtInt parameter.	Y/N	N
QueueTooHigh	Specifies whether the adapter will queue received messages in the adapter if their sequence number is higher than expected. If this parameter is set to N out-of-sequence messages received will be sent on to the output stream. If it is set to Y out-of-sequence messages will be queued in the adapter until such time as the next expected message is received.	Y/N	N
MaxQueueLength	If QueueTooHigh is enabled, specifies the maximum number of messages with out-of-order sequence numbers the adapter queues. The adapter continues to emit tuples in order of sequence number when the maximum queue length is exceeded. Queuing restarts when the queue length drops to MaxQueueLength/2. Discarding messages due to excessive queue lengths protects StreamBase Server from heap exhaustion at the expense of increased retransmission activity.	positive integer	10000
QueueCapacity	Specifies the maximum number of unprocessed messages the adapter holds in its input queue. This parameter is used to avoid heap exhaustion when the adapter requests large retransmissions on the recovery feed.	positive integer	10000
RetransRetryCount	The number of times the adapter attempts to recover the messages within a specific range of sequence numbers by sending retransmit requests. If the retry count is exceeded, the adapter logs an event indicating the messages in the range could not be recovered and continues processing the feed, starting with any queued messages.	positive integer	10000
RetransRetryWaitTime	The number of milliseconds the adapter waits between attempts to recover messages.	positive integer betwen 0 (no wait) and 1000 (one second).	100
split	Specifies to the adapter which message types (as denoted by either ControlClass or BusinessClass) should be split into multiple tuples if they contain repeating groups of fields.	Any valid message types, specified as a comma-separated list if more than 1 is desired.	—
Decrypt	Specifies to the adapter whether it should attempt to decrypt the private content portion of incoming feed messages for which the PrivateKeyIdentifier matches the one assigned to the adapter in its configuration.	Y/N	Y

Storage
PersistMessages	If set to N, no messages will be persisted to the log.	Y/N	Y
FileStorePath	Directory to store sequence number and message files. Only used if storetype=file.	valid directory for storing files, must have write access	—

Logging
FileLogPath	Directory to store logs. Only used if logtype=file.	valid directory for storing files, must have write access	—
FileLogHeartbeats	Controls whether or not heartbeat messages are logged. Only used if logtype=file.	Y/N	N
ScreenLogEvents	Log events to screen. Only used if logtype=screen.	Y/N	Y
ScreenLogShowIncoming	Log incoming messages to screen. Only used if logtype=screen.	Y/N	Y
ScreenLogShowOutgoing	Log outgoing messages to screen. Only used if logtype=screen.	Y/N	Y
ScreenLogShowHeartBeats	Filter heartbeats from output (both incoming and outgoing). Only used if logtype=screen.	Y/N	N

Sequence Number Reset
ResetOnconnect	Determines if sequence numbers should be reset to 1 when a session connects.	Y/N	N
ResetOnDisconnect	Determines if sequence numbers should be reset to 1 when a session terminates.	Y/N	N

Typechecking and Error Handling

The TBF Input adapter uses typecheck messages to help you configure the adapter in your StreamBase application. In particular, the adapter generates typecheck messages when an invalid or no configuration file is specified, when an invalid symbols file is specified, when neither the equity feed and nor reconciliation feed is enabled, or when the scheme for an enabled feed has not been configured.

The adapter generates messages on the status port during runtime under various conditions, including:

A STAMP session is created.
The adapter logs in to a recovery session.
A connection to the STAMP server fails.
An error occurs in processing an incoming STAMP message.
The adapter is suspended or resumed.

Suspend and Resume Behavior

When suspended, the TBF Input Adapter stops emitting tuples on its output ports.

When resumed, the adapter begins emitting tuples on its output ports once again.