Contents
The TIBCO StreamBase® SBD-to-SBD Output (Upstream) Adapter runs within an upstream application, sending tuples to a downstream application. It is suitable for situations in which multiple data producers — upstream applications — wish to enqueue data to a single consumer. The producers can be stopped and started independently without affecting the consumer.
The adapter is configured with the host name and port of the downstream application as well as the name of the stream, or streams to send data to. The adapter is configured by default to reconnect after a restart of the downstream application.
The following table explains the adapter properties that you can set in the Adapter Settings tab of the Properties view for an SBD2SBDOutput icon in the EventFlow canvas.
| Property | Data Type | Description |
|---|---|---|
| Downstream host | string | The host name or IP address of the downstream application. |
| Downstream port | string | The TCP port number of the downstream application. |
| Downstream stream | string |
The name of the first stream to write to in the downstream application. If no
stream with this name is present in the downstream application, or the
stream's schema is incompatible with the schema configured in the adapter,
the upstream application behaves as though the downstream application is
inaccessible. The Reconnect Interval property
determines whether the upstream application attempts to reconnect
periodically until a compatible stream is present.
To specify the names of the second and subsequent streams to write to, enter the stream names in separate rows in the Additional Streams tab of the Properties view. |
| Downstream container | string |
The name of the container in the downstream application that holds the
Downstream stream designated in the previous
field. The default entry is default.
|
| Reconnect Interval (secs) | int |
The period, in seconds, the upstream application waits between reconnection
attempts after the downstream application fails or no compatible stream is
present. Set this property to zero to disable reconnection; the default value
is 10 seconds. When reconnection is disabled, the downstream application must
be started first and must have a compatible output stream from which the
upstream application can read.
NoteWhen specifying a reconnection interval as part of a container connection to a remote host, specify the interval value in milliseconds, not seconds. See Remote Container Connection Parameters. |
| Enqueue buffer size (tuples) | int | The numbers of tuples to enqueue before tuples are sent to the downstream application. For a low-latency application, set the buffer size to zero. |
| Max enqueue buffer size (tuples) | int |
The maximum size of the enqueue buffer before tuples are dropped. Enter
-1 to disable this setting.
|
| Enqueue flush interval (msecs) | integer | The flush interval of the enqueue buffer in milliseconds. For a low-latency application, set the flush interval to a small non-zero number. Do not set it to zero, which causes the adapter to perform a busy loop and consume excessive CPU cycles. |
| Use SSL | check box | Select this check box if the connection to the downstream host uses secure SSL-based communication to the downstream server. |
| Authentication type | drop-down list |
Specifies the type of authentication to use when connecting to the downstream
server. Options in the drop-down list are: No
authentication, Basic authentication, and
Client certificate authentication.
|
| User name | string | For secure connections, the login name on the downstream host to authenticate as. Only one of User name or Keystore pathname can be specified. |
| Keystore file pathname | string | The path on the local machine to the file containing keystore information used for Client certificate authentication. |
| Password | string | The password for the User name or the keystore passphrase. |
| Schema matching | two-option radio buttons |
Identical structure required means that, in the
schemas of the outgoing and incoming streams, each field must match in data
type and sequence, but the field names do not have to match. For fields of
type tuple, their fields must also match in type and sequence.
Match field names, the default setting, means that fields in the outgoing stream are matched by name against fields in the incoming stream. Fields whose names match must have the same data type in both streams. Any fields in the outgoing schema whose names don't match the incoming schema are not streamed. Any fields in the incoming schema whose names don't match anything in the incoming schema are set to null.
For example, consider an outgoing stream with schema |
| Capture Transform Strategy | radio button | The strategy to use when transforming capture fields for this operator: FLATTEN or NEST. |
| Explicit URI (Advanced) | string | This option is for advanced users who have a standard StreamBase URI they use in other parts of the StreamBase application. Use this option to paste in a full StreamBase URI to the downstream application, including security connection information. If this field is used, the following fields must be blank: Downstream host, Downstream port, Downstream stream, Downstream container, User name, Keystore file pathname, and password. |
| Enable event port | check box |
Select this check box to enable an output port that receives connection-up
and connection-down events. The schema of the event output port is as
follows:
|
| Connect on adapter init | check box | Selected by default. When selected, or no control port is enabled, the adapter attempts to connect to the other server during adapter start-up. When cleared, the adapter connects to the other server when a connect tuple is received on its control input port. |
| Enable control port | check box | Cleared by default. When selected, an additional input port is created, used to control and request status of the adapter. See below for more on this input port |
| Low Latency | check box | Cleared by default. When selected, the adapter configures itself to minimize data latency by disabling buffering, Nagle algorithms, and so on. |
| Log Level | drop-down list | Controls the level of verbosity the adapter uses to issue informational traces to the console. This setting is independent of the containing application's overall log level. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE, and ALL. |
Low Latency, Enqueue buffer size and Enqueue flush interval are three properties that affect the latency and throughput of the StreamBase to StreamBase output adapter.
Enabling Low Latency configures the adapter in its lowest latency mode. Enqueue buffer size and Enqueue flush interval are ignored in this mode.
For maximum throughput, set the Enqueue buffer size property large enough to reduce the per-packet overhead to a minimum. Values of 10 to 1000 are common; the default is 100. Set the Enqueue flush interval property high enough so that, in general, the enqueue buffer size is reached, and the data is sent, before the flush interval expires. For high throughput applications, the flush interval is rarely hit.
For minimum latency, set the Enqueue buffer size property to 0. This forces each tuple to be sent immediately, in line. With a buffer size of zero, the Max enqueue buffer size and Enqueue flush interval properties are not used. This results in the lowest latency configuration, but comes at the cost of higher system overhead. Tuples are sent one per network packet, which increases network overhead and increases CPU usage on both the client and server. For low latency applications, see Java System Properties for more information on configuring clients and servers.
Setting the Enqueue buffer size property to 1 or higher causes tuples to be placed on a queue, and a network writer thread performs the actual network writes. The latency difference between an Enqueue buffer size of 0 and 1 is only the cost of adding the tuple to a queue and notifying the write thread. The disadvantage of an Enqueue buffer size of 0 is that if the downstream dequeuer is slow, the application must pause when network backup occurs, as the network write occurs in line.
The StreamBase to StreamBase output adapter has an optional event output port defined above.
For the default stream, and each additional configured stream, an input port is added to the adapter.
If you select the Enable control port option, an
input port is added to the adapter for use as a control port. The first field of
the control port schema must be a string field named command. This port is used to send the following commands to the
adapter:
-
connect— The adapter tries to connect to the configured downstream host. If configured to reconnect, it continues to try to connect at the configured reconnect interval until it succeeds. -
disconnect— The adapter disconnects from the downstream host. It does not try to reconnect, even if a reconnect interval is configured. -
status— The adapter emits a status tuple on the event port, if configured.
Typechecking fails if the Downstream Host or
Downstream Stream properties are empty or if the
Downstream Port property contains an illegal TCP port
number (unless the Explicit URI field is used instead).
The behavior in response to errors depends upon the setting of the Reconnect Interval property. When configured to reconnect, the upstream application polls periodically until the downstream application is accessible and contains an output stream with an identical name and schema. Otherwise, the upstream application emits an error message and makes no further attempts to connect to the downstream application.
The StreamBase to StreamBase output adapter throttles dropped-tuple warnings as follows:
-
The first dropped tuple generates an immediate warning.
-
Subsequent dropped tuples are counted, but not immediately reported.
-
When at least one tuple has been dropped, and at least 10 seconds have elapsed since the previous dropped-tuple warning, a new warning is generated the next time a tuple is processed, whether that tuple is dropped or not.
-
Dropped-tuple warnings indicate the number of tuples dropped and the number of seconds elapsed (normally 10) since the last warning, or in the case of the first warning, since the adapter was started.
-
No warnings are generated if no tuples are dropped since the last warning.
-
A warning can cover more than 10 seconds if the adapter drops a tuple and then becomes idle.
When suspended, the adapter stops sending tuples to the downstream application. Note that doing so may cause the application to drop the connection with the downstream application, resulting in lost tuples. Upon resuming, the adapter reconnects to the downstream application, if necessary and configured to do so, and begins sending tuples. If the adapter is configured not to reconnect with the downstream application and the connection is dropped during the suspension, the application will not enqueue tuples to the downstream application upon resume.
However, when the container enqueue (dequeue) control is set explicitly to ENABLED from any other state (DISABLED, DROP_TUPLES, or even ENABLED), the StreamBase to StreamBase adapter then starts (that is, enters the STARTED state), regardless of the adapter's setting.