Using the Iterate Operator

Introduction

  Use the Iterate operator to emit one tuple for each element in a list. You specify a field of type list in an incoming stream, and the operator emits one tuple for each element in that field's list. Since list fields can contain a varying number of elements for each tuple, this operator resolves each incoming tuple into multiple outgoing tuples.

By default, all elements of the list value are output, in list order, one at a time. You can stop emitting elements when a predicate condition becomes false. The condition can be based on a running aggregation of previous list elements. You can also selectively emit those elements that satisfy a filter predicate. You can specify or define fields to output using the field grid on the Output Settings tab, in a similar fashion to the Map operator

Note

The Iterate operator acquired its output field grid in StreamBase Studio 7.4. Previously, values for each iteration were placed in a prepended output field, the name of which the Output Element Field Name parameter on the Operation Settings tab specified. This parameter was removed in favor of the flexibility of a field grid, which enables any number of output fields to be defined. When running or editing older EventFlow applications that use Iterate operators, the current version upgrades applications by adding a field with the specified Output Element Field Name to the output field grid using the expression each.element to capture the current element.

For example, in incoming tuple 1, a list(double) field named Bids might have six elements. For each input tuple the Iterate operator receives, it will output six tuples (unless it has a filter expression that suppresses some), having the same fields as the incoming tuple, subject to additional expressions specified in the Output Settings field grid. For details, see Properties: Operation Settings Tab.

For example, the field grid could define an output field named oneBid using the expression each.element.Bids. That field will contain single elements of the Bids list, one at a time in list order, for each outgoing tuple and be be of data type double.

Then, in incoming tuple 2, list Bids might have three elements. The Iterate operator receives incoming tuple 2 and outputs three tuples containing oneBid fields, and so on.

The StreamSQL equivalent of the Iterate operator is the FOREACH clause of the SELECT statement. See SELECT Statement in the StreamSQL Guide. Note that iteration conditions are not supported in SSQL, but iteration filters are (as WHERE clauses).

Properties: General Tab

Name: Use this field to specify or change the component's name, which must be unique in the application. The name must contain only alphabetic characters, numbers, and underscores, and no hyphens or other special characters. The first character must be alphabetic or an underscore.

Enable Error Output Port: Select this check box to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports and Error Streams to learn about Error Ports.

Description: Optionally enter text to briefly describe the component's purpose and function. In the EventFlow canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Properties: Operation Settings Tab

The properties view for the Operation Settings tab has the following appearance.

Use the Operation Settings tab to specify the following parameters:

Property Description Allowable Expression Entries
Iterate Over The name of a field of data type list in the incoming stream, or an expression that resolves to a field of type list. (required) list(T) or any expression whose result is of type list
Iterate Condition A boolean regular or aggregate expression which, when it evaluates to true for a list element, continues to iterate the input list. If it is false, iteration terminates for that tuple, like a While loop. If left empty, all elements are iterated over.
  • Input tuple fields

  • The each item

  • Dynamic variables

  • Constants and Parameters

  • Simple and Aggregate functions

Iterate Filter A boolean regular expression which, when it evaluates to true for a list element, includes the element in the output. If it is false, that element is skipped and the next list element is processed. If left empty, all elements are included, subject to the Iterate Condition. That is, te Iterate Filter is applied after the Iterate Condition predicate, if any, has been met.
  • Input tuple fields

  • The each item

  • Dynamic variables

  • Constants and Parameters

  • Simple functions

You can use Iterate operators to eliminate list items that are null or fail to meet some criterion by applying a filter, or transform list item values, and then use an aggregate operator to assemble the tuples that the Iterate operator emits into a new list having the same or a different field name. For more information, see the Iterate Operator Sample and aggregate list function descriptions.

Specify an Iterate Condition if the operator should stop processing a list as soon as one or more logical criteria are met. For example, you might limit output to the first n list items or stop iterating when a flag in some field has or does not have a certain value.

Properties: Output Settings Tab

The Input Field grid in the Output Settings tab is collapsed by default. When you expand it, the view has two grids, as illustrated here:

In the illustration above (taken from the sample application Iterate.sbapp), input tuples with schema {symbol string, name string, volume int, price double} have been aggregated into a list that the Iterate operator deconstructs into tuples, replacing volume and price with the field StockCost (using the expression each.element.Volume*each.element.Price). No input fields are passed directly to output, but the additional expressions each.element.Name and each.element.Symbol insert both string fields into the output stream.

Note

The expressions for StockName and StockSymbol are shown to illustrate how to use the each construct. Unless you need to change their names, simply include fields like Name and Symbol in the output using the Input Field grid, as described previously.

Properties: Concurrency Tab

Use the Concurrency tab to specify the use of parallel regions for this instance of this component. Consider selecting the parallel regions check box if this component instance is long-running or compute-intensive, can run without data dependencies on other StreamBase components, and would not cause the containing module to block while waiting for a thread to return. In this case, you may be able to improve performance by selecting this option. This option directs StreamBase Server to process this component concurrently with other processing in the application. The operating systems supported by StreamBase automatically distribute the processing of threads across multiple processors.

Caution

The parallel regions setting is not suitable for every application, and using this setting requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.