ETP Adapters

Note

The ETP adapters are provided for Microsoft Windows systems only.

Introduction

The TIBCO StreamBase® Adapter for ETP allows StreamBase to interact with data from an ETP (Energistics Transfer Protocol) v1.1 server. The adapters in this set include:

  • The ETP Control adapter, which is used to control the connection with the ETP server (connection, disconnection, server status, and so on).

  • The ETP Store Get adapter, which is used to retrieve values from ETP nodes.

  • The ETP Discovery adapter, which is used to traverse the ETP server's node hierarchy and retrieve metadata for these nodes.

  • The ETP Streaming adapter, which is used to subscribe to value nodes and get continuous updates when the nodes' values change, as well as perform range requests for such values.

Placing an ETP Adapter on the Canvas

The ETP adapters are members of the Java Operators group in the Palette view in StreamBase Studio. Select the operator from the Insert an Operator or Adapter dialog, following these steps:

  1. Open the Insert an Operator or Adapter dialog with one of the following methods:

    1. Drag the Adapters, Java Operators token from the Operators and Adapters drawer of the Palette view to the canvas.

    2. Click on the canvas where you want to place the operator, then invoke the keyboard shortcut O V.

    3. From the top-level menu, invoke Insert>Operator>Java.

  2. In the search field, enter ETP to narrow the list of adapters.

  3. Select the ETP adapter of interest and click OK.

Prerequisites

In order run correctly, the adapters require that the Microsoft .NET Framework 4.8 or higher be installed on the machine.

How the Adapters Work

All adapters have one input stream to receive commands. The schema for this command stream varies depending on the adapter type; each schema is described in a section below.

All adapters have an output stream to communicate status events (such as when an error occurs during the execution of a command). The schema for this stream is identical for all ETP adapter types (save for its inputTuple tuple field, whose schema will vary according to the operator's input stream's schema) and is described in a section below.

With the exception of the Control adapter, all adapters also have a second output port to emit the results of executing commands. For example, the Store Get adapter uses this port to emit the value(s) it reads from the server.

Finally, the Streaming adapter has an additional port to emit metadata collected for requested nodes.

Configuration

The different ETP adapters share a connection to the same server, provided they are configured to do so. Each adapter lists the servers available in a combo box (see ETP Adapter Properties). The list's values are specified in a dedicated section of the application's adapter configuration file. Here is an example of such a section, containing all supported settings (long lines wrap to the next, for clarity):

name = "ETP.conf"
type = "com.tibco.ep.streambase.configuration.adapter"
version = "1.0.0"
configuration = {

  // An adapter group type defines a collection of EventFlow adapter configurations, 
  // indexed by adapter type.
  AdapterGroup = {

      // A collection of EventFlow adapter configurations, indexed by adapter type. 
      // This object is required and // must contain at least one configuration.
      adapters = {

        // The root section for an EventFlow adapter configuration.
        etp = {

          // Section list. This array is optional and has no default value.
          sections = [ 

            // A configuration for an EventFlow adapter named section.
            {

            // Section name. The value does not have to be unique; that is, you can have 
            // multiple sections with the same name in the same array of sections. This property
            // is required.
            name = "server-definition"

            // Section of properties. All values must be strings. This object is optional 
            // and has no default value.
            settings = {
              id = "MyETPServer"
                            
              // The ETP server to which to connect
              server-url = "ws://localhost/Witsml.Web/api/etp"
                            
              // 0 to not re-attempt connection when the attempt fails, -1 to continuously re-attempt
              num-reconnection-retries = "5"
              reconnection-retry-interval-ms = "10000"

              // Credentials to use for connection (leave empty for anonymous connections)
              // Note that you can encipher "password" using the "epadmin encrypt secret" command
              username = ""
              password = ""
                            
              // Proxy-related setting. Leave empty when not using a proxy to connect to ETP server
              // Note that you can encipher "proxy-password" using the "epadmin encrypt secret" command
              proxy-host = ""
              proxy-port = ""
              proxy-username = ""
              proxy-password = ""
           }
         }
       ]
      }
    }
  }
}

A best practice is to define your endpoints in this file before placing operator instances on the canvas, so that the lists are already available in the Properties view and the operators can be configured right away.

ETP Adapter Properties

This section describes the properties you can set for the adapters, using the various tabs of the Properties view in StreamBase Studio.

In following section tables, the Property column shows each property name as found in the one or more adapter properties tabs of the Properties view for this adapter.

Use the StreamSQL names of the adapter's properties when using this adapter in a StreamSQL program with the APPLY JAVA statement.

ETP General Tab

Name: Use this required field to specify or change the name of this instance of this component, which must be unique in the current EventFlow module. The name must contain only alphabetic characters, numbers, and underscores, and no hyphens or other special characters. The first character must be alphabetic or an underscore.

Operator: A read-only field that shows the formal name of the operator.

Class name: Shows the fully qualified class name that implements the functionality of this operator. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this operator starts.

Enable Error Output Port: Select this check box to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally enter text to briefly describe the component's purpose and function. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

ETP Operator Properties

The ETP adapters have the following properties, with the exceptions noted for some properties only on particular adapters:

Property Data Type Default Description
ETP Server Definition combo box Cleared

REQUIRED. Specifies the ID of the ETP server to which to connect, as defined in the adapter configuration file.

Connect on Startup check box Cleared CONTROL ADAPTER ONLY. Attempt to connect to the server when the application starts.
Log Level drop-down list INFO Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level will be used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.

Cluster Aware Tab

Use the settings in this tab to allow this operator or adapter to start and stop based on conditions that occur at runtime in a cluster with more than one node. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with TIBCO Streaming releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.

Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Input Ports

The adapters have a single input port which is used to receive commands. The schema for this port is different for each adapter; those schemas are described in this section.

ETP Control Adapter Input Port

This port is used to connect or disconnect from the configured server, and to get the current connection status. Results are issued on the adapter's Status output port.

Field Name Field Type Description
command string REQUIRED. The command to execute. One of connect, disconnect, getconnectionstatus. Values are case-insensitive.

ETP Discovery Adapter Input Port

The Discovery adapter allows your application to traverse the data hierarchy of the ETP server. Using this port you can retrieve the metadata of the child nodes of the node specified in the uri field. This metadata includes the child node's uri, content type and several other items. For any given query, one tuple is emitted on the Discovery adapter's metadata output port (see the description of this output port below) for every child node found.

Field Name Field Type Description
uri string REQUIRED. Contains the URI of the node for which to retrieve child node metadata. If no value is provided, the root node of the server will be queried.

ETP Store Get Adapter Input Port

This port instructs the adapter to retrieve the current value of the specified ETP node. The retrieved data will be emitted on the results output port of the adapter (see the description of this output port).

Field Name Field Type Description
uri string REQUIRED. Contains the URI of the node from which to retrieve data.

ETP Streaming Adapter Input Port

This port is used to start (and stop) monitoring one or more data nodes for changes in their value. Periodical updates are issued by the adapter for each of these subscriptions until the subscription is removed via a StreamingStop command to this port. Multiple subscriptions may be active at the same time.

Before a streaming session can be initiated, a Start command must be sent to the server via the adapter's input port, optionally specifying values for the maxDataItems and maxMessageRate fields.

ETP servers come in two configurations: Simple and Non-Simple. Simple streaming servers start sending channel updates for all values immediately after the Start command is received. For Non-Simple servers, a Describe command must also be sent, followed by the StreamingStart command to specify exactly which data points to stream.

Finally, you can use this port to issue ETP range requests, via the RangeRequest command.

Field Name Field Type Description
command string REQUIRED. One of Start, Describe, StreamingStart, StreamingStop, RangeRequest
maxDataItems int Start command only — OPTIONAL. The maximum number of items to return in an update.
maxMessageRate int Start command only — OPTIONAL. The rate at which updates are emitted. That is, the minimum amount of time (in milliseconds) between updates.
uris list<string> Describe command only — REQUIRED. The URIs of the nodes to describe.
channelInfos list<tuple> StartStreaming command only — REQUIRED. The information on each channel to start streaming.
^ channelID long The ID of the channel to stream.
startIndex double The index at which to start streaming.
receiveNotifications boolean Whether to receive updates (true) or just a single snapshot (false).
channelIDs list<long> StopStreaming command only — REQUIRED. The IDs of the nodes to stop streaming.
rangeInfos list<tuple> RangeRequest command only — REQUIRED. The information for each node for which to request the range.
^ channelIDs list<long> The IDs of the nodes to request.
startIndex long The index at which to start.
endIndex long The index at which to stop.

Output Ports

Every adapter (except the Control adapter) has an output port to emit tuples resulting from executing queries. In addition, every adapter has an output port to receive status events. Finally, the Streaming adapter also has a third port to emit metadata tuple.

Status Output Port (Common to all ETP operators)

The status port emits tuples that describe the status of processing each input tuple. The schema of the output tuple is as follows:

Field Name Field Type Description
status string The type of status represented by this tuple (Connected, Disconnected, Success, Error, and so on).
message string A description of the event.
exception string If this event was generated as a result of an exception, this field describes the exception. Otherwise, it is left null.
time timestamp The time at which this status tuple was generated.
serverInfo tuple If applicable, this tuple will contain more information on the server related to this status event. When this tuple is set, its serverID will always be non-null but all other values may be left empty depending on what information is available on it at the time the status tuple is generated.
^ serverID string The ID of the server, as it appears in the adapter's configuration file (see Configuration above).
serverURL string The server's URL.
sessionID string The ID of the current session with the server.
applilcationName string The server's application name
applicationVersion string The server version.
connectionStatus string Typically used in reply to the Control adapter's GetConnectIonStatus command. One of Connected, Disconnected.
supportedProtocols list<string> A list of the ETP protocols that this server supports. Typically set for a Connected event.
supportedObjects list<string> A list of the ETP data types (expressed as MIME strings) that this server supports. Typically set for a Connected event.
inputTuple tuple The input tuple that caused this status tuple to be emitted, if any.

ETP Discovery Adapter Results Output Port

The Discovery adapter allows your application to traverse the data hierarchy of the ETP server. This port emits tuples describing the metadata of a node on the server. This metadata includes the node's name, uri and content type, whether it has child nodes and so forth. One tuple will be emitted on this port for every child node found at the uri provided in the command input tuple.

Field Name Field Type Description
header tuple The header of the ETP message represented by this tuple.
^ protocol int  
messageType int  
correlationId long  
messageId long  
messageFlags int  
resource tuple The description of this node.
^ uri string The uri of this resource.
contentType string The content type (typically in MIME format) of this resource.
name string The name of this resource.
channelSubscribable bool Indicates that this resource is the source of one or more channels for streaming. Specifically, the URI in this record can be sent in a Describe command in the Streaming adapter with an expectation of receiving one or more channels.
resourceType string The type of this resource.
hasChildren int Indicated the number of children for this resource:
  • -1: Child count is unknown

  • 0: No Children

  • Positive value: Child count

uuid string  
lastChanged int  
objectNotifiable boolean Whether this node is streamable.
customData list<tuple> The list of key-value pairs representing any additional data for this resource.
^ key string  
value string  
Input tuple The command tuple sent to the input port that triggered this result.

ETP Get Adapter Results Output Port

The Get adapter allows your application to query the current value of a node on the ETP server. All retrieved values are given as strings unless such a conversion is not possible, in which case the value will be given as a blob.

Field Name Field Type Description
header tuple The header of the ETP message represented by this tuple.
^ protocol int  
messageType int  
correlationId long  
messageId long  
messageFlags int  
dataObject tuple The data retrieved for this node.
^ resource tuple The value of this node.
^ uri string The uri of this resource.
contentType string The content type (typically in MIME format) of this resource.
name string The name of this resource.
channelSubscribable bool Indicates that this resource is the source of one or more channels for streaming. Specifically, the URI in this record can be sent in a Describe command in the Streaming adapter with an expectation of receiving one or more channels.
resourceType string The type of this resource.
hasChildren int Indicates the number of children for this resource:
  • -1: Child count is unknown

  • 0: No Children

  • Positive value: Child count

uuid string  
lastChanged int Time
objectNotifiable boolean Whether this node is streamable.
customData list<tuple> The list of key-value pairs representing any additional data for this resource.
^ key string  
value string  
contentEncoding string The data encoding scheme used by this node
dataAsBytes blob The data retrieved, as a blob. This is null unless dataAsString is null.
dataAsString string The data retrieved, as a string. If the data cannot be represented as a string, this field is left null and dataAsBytes is set instead.

ETP Streaming Adapter Output Ports

Use the Streaming adapter to retrieve metadata and data values for a given list of ETP nodes. The Describe command of this operator (see the description of this input port) will issue the requested metadata to the Metadata Results output port, while the StartStreaming and RangeRequest commands will output data to the Data Results output port.

ETP Streaming Adapter Metadata Results Output Port

Describe commands set to this operator will issue tuples to this port.

Field Name Field Type Description
header tuple The header of the ETP message represented by this tuple.
^ protocol int  
messageType int  
correlationId long  
messageId long  
messageFlags int  
channels list<tuple> The list of metadata records retrieved.
^ channelID string The ID of the channel represented by this metadata record.
channelName string  
channelURI string  
dataType string  
uom string Unit of measure.
startIndex long  
stopIndex long  
description string  
status string  
contentType string  
source string  
measureClass string  
uuid string  
domainObject tuple  
^ resource tuple  
^ uri string  
contentType string  
name string  
channelSubscribable boolean Whether this node is streamable.
resourceType string  
hasChildren int Indicates the number of children for this resource:
  • -1: Child count is unknown

  • 0: No Children

  • Positive value: Child count

uuid string  
lastChanged long  
objectNotifiable boolean  
customData list<tuple>  
^ name string  
value string  
contentEncoding string  
dataAsBytes blob  
dataAsString string  
indexes list<tuple>  
^ uri string  
indexType string  
uom string  
depthDatum string  
direction string  
mnemonic string  
description string  
scale string  
timeDatum string  
customData string  
^ name string  
value string  
customData list<tuple>  
^ name string  
value string  

ETP Streaming Adapter Data Results Output Port

StartStreaming and RangeRequest commands sent to this adapter will output their data to this port. The Streaming adapter allows your application to retrieve the data for one or more streamable nodes. Additionally, when using the StartStreaming command, if the channelInfos.receiveNotifications field (see the description of this input port) of the input tuple used to request these values was set to true, tuples will be emitted periodically to report on the monitored nodes' values. Thus, by controlling the value of the channelInfos.receiveNotifications field you can use this adapter to act either as a way to retrieve the value of multiple nodes in one call (channelInfos.receiveNotifications = false) or as a way to subscribe to the node(s)' ongoing changes (channelInfos.receiveNotifications = true).

The data field of each tuple emitted from this port is a list of all the nodes for which data was retrieved, their value, any optional attributes, and the like.

Field Name Field Type Description
header tuple The header of the ETP message represented by this tuple.
^ protocol int  
messageType int  
correlationId long  
messageId long  
messageFlags int  
data list<tuple> The data of the nodes retrieved.
^ channelID long ID of the node represented by this entry.
indexes list<long>  
value string The value of this node, represented as a string.
valueAttributes list<tuple> The list of key-value pairs representing any additional attributes for this resource.
^ attributeID string  
attributeValue string  

Data Conversion

Because different nodes may contain values of different types, the various ETP operators represent each value as a string, but also provide metadata such as data type to inform on their actual representation in the ETP server. Using this information, the StreamBase application can parse the string values into more representative types.

Typechecking and Error Handling

Typechecking fails if the input schema does not contain all the required fields, or if a valid ETP Server Definition has not been specified.

Suspend and Resume Behavior

Upon suspension, the ETP adapters finish processing the current tuple, outputs the result tuple(s), then pauses waiting for input.

Upon resumption, the ETP adapters continue processing with the next input tuple.

In either case, any database connection is not modified.

ETP Adapter Sample

The StreamBase installation includes a sample demonstrating the use of this operator. To load the sample in StreamBase, select File>Import Samples and Community Content and look under the IoT section for an entry called ETP Adapters.

Back to Top ^