Using the Matrix Operator

Introduction

The TIBCO Streaming® Matrix Operator is used to gather tuples over various styles of output types such as over time or by selected values. The purpose of this operator is to create a matrix (list of tuples) of which the tuples fields are the columns of the matrix.

Matrix Processing Order

The Matrix operator processes the input tuple in the following order:

  • Evaluates all missing data rows and replaces values as specified

  • Evaluates the output expression and adds the tuple to the matrix

  • Checks whether the matrix should be emitted downstream

  • Evaluates the Null Value Expression on null fields of each tuple in the matrix

  • Emits the matrix downstream

Matrix Operator Properties

This section describes the properties you can set for this operator, using the various tabs of the Properties view in StreamBase Studio.

General Tab

Name: Use this required field to specify or change the name of this instance of this component, which must be unique in the current EventFlow module. The name must contain only alphabetic characters, numbers, and underscores, and no hyphens or other special characters. The first character must be alphabetic or an underscore.

Operator: A read-only field that shows the formal name of the operator.

Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.

Enable Error Output Port: Select this check box to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally enter text to briefly describe the component's purpose and function. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Operator Properties Tab

Property Description
Output Type
  • Row Added — A matrix is emitted for each tuple that is added so long as the minimum number of rows has also been reached.

  • Any Selected — A matrix is emitted when the value of any of the fields marked as Selected on the Output tab have a value change as compared to the previous row of the matrix. If there is no previous row a matrix is not emitted.

  • All Selected Fields — A matrix is emitted when the value of all the fields marked as Selected on the Output tab have a value change as compared to the previous row of the matrix. If there is no previous row a matrix is not emitted.

  • Interval — A tuple is emitted at a set interval. Each field of the output tuple is replaced by any non-null result of evaluating the input tuple against the expression given in the Output Tab.

Sort Field This field is used to sort the tuples in the matrix. If no field is selected, the input tuples are added to the matrix in the order in which they are received. If the input tuples sort field equals NULL then the tuple is sorted to the end of the matrix. If more than one row has a NULL value for the sort field, then all NULL rows sort to the end and use the arrival order as a secondary sort.
Time Unit The time unit used when calculating intervals.
Interval The number of Time Unit values between intervals. For example, 1 second or 1 hour depending on the Time Unit selected.
Min. Number Of Rows The minimum number of rows in the matrix before it emits downstream, depending on the Output Type selected.
Max Number Of Rows The maximum number of rows stored in the matrix. If a tuple arrives and the matrix grows larger than the maximum number of rows, then rows are removed from the matrix depending on the Sort Field selected. If no sort is done, the oldest tuples are removed.
Clear Matrix On Emit If enabled, the matrix is cleared when the matrix is emitted downstream.
Log Level Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.

Missing Data Tab

The Missing Data tab is used to check incoming tuples against an expression; if the expression evaluates to true then replace the field with the given replacement value. This missing data replacement check is done before the input tuple is evaluated further inside the operator. By default, no field values are replaced. You can use the smart fill option to evaluate the current input schema and fill the table with reasonable values.

Column Description
Field Name The name of the input field to replace its value if the expression evaluates to true.
If Below Is True: This column is used to enter the expression which is evaluated against the input tuple to determine if the field given in the first column should be replaced by the value given by the 'Then Replace with' column
Then Replace with: This column is the expression for which the results will be used to replace the Field Name field of the input tuple before any processing occurs. If the expression is empty, a null is used.

Output Tab

The Output tab is used to determine what fields create the tuple to add to the matrix. The output can have any number of fields regardless of the number of input fields. Each row of the output fields table is a field to add to the matrix, which will be output and the expression column determines what the value of that field will have. The smart fill option can be used to evaluate the current input schema and fill the table with reasonable values.

Column Description
Field Name The field name in the output tuple.
Expression The expression that will fill the value of this field in the output tuple. This may be an aggregate expression if the output type is set to aggregate interval.
Null Value Expression Directly before the output tuple is sent downstream, each field is checked for null. If it is null then this expression is evaluated and the resulting value replaces the current null value. If this expression is empty no replacement is done and the null value remains.
Selected If the output type is either Any Selected or All Selected then this column is used to select which fields are used to determine if the field should be used when determining if the matrix should be emitted downstream.

Cluster Aware Tab

Use the settings in this tab to allow this operator or adapter to start and stop based on conditions that occur at runtime in a cluster with more than one node. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with TIBCO Streaming releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.

Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Data Input Port

The Data Input port can consist of any schema and is used by the operator's expressions to evaluate and matrix output tuples.

Data Output Port

The Data Output port contains a list of tuples for which the schema is described by the Output Tab settings.