Defining Input Streams

Introduction

Input streams are generally shown on the left side of an EventFlow canvas. They define an entry point into a StreamBase application, and declare the starting schema of inbound data. You can define one or more input streams in an application.

The name and schema you assign to an input stream are used by StreamBase Server to determine whether data from clients and external sources should be enqueued onto that stream. StreamBase Server can listen for requests from a client source, such as the Manual Input View in the SB Test/Debug perspective, a feed simulation, or a producer client that you create based on the StreamBase Client API. The data can also arrive via an adapter that converts third-party protocols into the StreamBase protocol.

Starting with StreamBase 7.2.6, the icons for input streams that are an implementation of a StreamBase interface are shown with a boxed uppercase I in the lower right corner of the icon's tile. Input streams that are used to update dynamic variables are shown with (x)= in the upper right of the icon.

The rest of this page describes the actions you can take on each tab of the input stream Properties view.

Properties: General Tab

Name: Use this field to specify or change the component's name, which must be unique in the application. The name must contain only alphabetic characters, numbers, and underscores, and no hyphens or other special characters. The first character must be alphabetic or an underscore.

Always expose Streams for Enqueue: Designates that this input stream is to be available for enqueuing, no matter how deeply the module that contains this stream is nested in a complex application.

Description: Optionally enter text to briefly describe the component's purpose and function. In the EventFlow canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Properties: Edit Schema Tab

Starting with StreamBase 7.1.0, schemas with no defined fields are supported. See Using Empty Schemas for details.

To define the input stream's schema, follow these steps:

  1. Use the Fields grid to specify the schema type:

    Named schema

    Use the drop-down list to select the name of a named schema previously defined in or imported into this module. The drop-down list is empty unless you have defined or imported at least one named schema for the current module.

    When you select a named schema, its fields are loaded into the schema grid, overriding any schema fields already present. Once you import a named schema, the schema grid is dimmed and can no longer be edited. To restore the ability to edit the schema grid, re-select Private Schema from the drop-down list.

    Private schema

    Populate the schema fields using one of these methods:

    • Define the schema's fields manually, using the Add button to add a row for each schema field. You must enter values for the Field Name and Type cells; the Description cell is optional. For example:

      Field Name Type Description
      symbol string Stock symbol
      quantity int Number of shares

      Field names must follow the StreamBase identifier naming rules. The data type must be one of the supported StreamBase data types, including, for tuple fields, the identifier of a named schema and, for override fields, the data type name of a defined capture field.

    • Add and extend a parent schema. Use the Add button's Add Parent Schema option to select a parent schema, then optionally add local fields that extend the parent schema. If the parent schema includes a capture field used as an abstract placeholder, you can override that field with an identically named concrete field. Schemas must be defined in dependency order. If a schema is used before it is defined, an error results.

    • Copy an existing schema whose fields are appropriate for this component. To reuse an existing schema, click the Copy Schema button. (You may be prompted to save the current module before continuing.)

      In the Copy Schema dialog, select the schema of interest as described in Copying Schemas. Click OK when ready, and the selected schema fields are loaded into the schema grid. Remember that this is a local copy and any changes you make here do not affect the original schema that you copied.

      The existing schema can be from a system stream, or from any named or unnamed schema defined in the current module or in another application in your workspace. You can also select a CSV text file and populate a schema with its column headers. Studio will attempt to infer data types from the first few rows of values, and you can override the types it identifies. Currently, auto-detection of int, double, boolean, string, timestamp and tuples are supported, but not lists or functions. When indicating tuples, the CSV header must identify subtuples with dot notation, for example as stock.symbol, stock.price.

    Use the Remove, Move Up, and Move Down buttons to edit and order your schema fields.

  2. You can optionally document your schema in the Schema Description field.

Properties: Advanced Tab

The Advanced tab has two controls:

Container Connection Field

Use the Container connection field to specify a container connection to another module in another container. When so connected, this input stream receives tuples from the specified output stream in the specified container. This field is one of several methods you can use to specify a container connection. See Container Connections to understand the differences between, and the advantages and disadvantages of each method.

The Container connection field takes a StreamBase expression that resolves to the qualified name of an output stream in another container. A qualified stream name follows this syntax:

container-name.stream-name

When using the Advanced tab for an input stream, you are already in the destination stream, so you only need to specify the qualified name of the source stream.

You must express the container-qualified stream name as a StreamBase expression. While editing the expression, you can type Ctrl+Space to invoke StreamBase's expression-completion feature to help write your expression. Click anywhere in the field, then click the three dots on the right side of the field to open an expression editing window that allows more editing room for multi-line expressions.

To express a known container and stream name, enclose the qualified name in single or double quotes to render the name as an expression.

The container connection expression can optionally include variables, constants, and expression language functions that resolve to a qualified container and stream name. However, remember that the container connection is established at container start time, so your container connection expression cannot reference tuple field names or contents, since no tuples have yet passed through at container startup. For the same reason, the expression cannot resolve any value of dynamic variables, including the default value.

The following example container connection expression specifies a connection to an output stream named OutgoingBids in an application named BidCollector.sbapp that runs in a container named bids:

"bids.OutgoingBids"

Each container connection must be expressed only once for a connected pair of containers. Do not express a container connection in the output stream of the sending container and again in the input stream of the receiving container. You can express the connection in either the sending or receiving container to the same effect. Do not specify the container connection in both places.

The schemas of both the sending and receiving sides of a container connection must match exactly. See Schema Matching for Container Connections for details.

If you precede the container-name.stream-name expression with colon-equals, the container connection becomes synchronous, as described in Synchronous Container Connections. For example:

":=bids.OutgoingBids"

See Syntax for Expressing Container Connections for a discussion of other container connection options, including connecting to streams on separate servers. You can use this field to specify any type of container connection that is appropriate for this input stream and for your application as a whole, including CSV and JMS connections. However, Studio cannot typecheck and validate your container connection syntax. The actual connection is not attempted until you run the application, either in Studio or at the command line.

Important

There are restrictions on the order in which you start containers that have container connections, both at the command line and in Studio. See Container Start Order for details.

See Studio: Connections in Stream Property Advanced Tabs on the Container Connections page for further information.

Starting with StreamBase 7.2.6, the icons for input streams show an overlay decoration on their upper back corner to indicate that this stream initiates a container connection to another stream by means of the Advanced tab's Container connection field.

Streams on the other side of the connection in the receiving container are not decorated. Decorations are not applied for connections specified to start at runtime in a deployment file, in a Studio launch configuration, in StreamBase Manager, or from the command line with the sbadmin command.

Container Connection Filter Expression Field

When specifying a container connection in this tab, you can also specify a predicate expression filter to limit the tuples that cross to the destination container. The expression to enter must resolve to true or false, and should match against one or more fields in the schema of the connection's stream. Enter only the filter expression, such as tradeID % 2 == 0