Contents
The Spotfire Streaming Field Gather Operator is used to gather fields of the input tuple over various styles of output types, such as over time or by selected values. The purpose of this operator is to gather disparate field point values into a filled tuple.
The Field Gather operator processes the input tuple in the following order:
-
Evaluates all missing data rows and replaces values as specified.
-
Evaluates the output expression and updates the output tuple with the results.
-
Checks whether the output tuple should be emitted downstream.
-
Evaluates the Null Value Expression on null fields of the output tuple.
-
Emits the tuple downstream if all fields pass the
Suppress Row If Nullcheck.
This section describes the properties you can set for this operator, using the various tabs of the Properties view in StreamBase Studio.
Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.
Operator: A read-only field that shows the formal name of the operator.
Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.
Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.
Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.
Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.
| Property | Description |
|---|---|
| Output Type |
|
| Timestamp Field | The Timestamp Field is used when Output Type is either Aggregate Interval or Interval. The field is used as the time during interval calculations. If a field is selected, then the first input tuple is used as
the starting time for the intervals. If the tuples time field is null, then now() is used by default. You can use the Missing
Data tab to change this default.
|
| Time Unit | The time unit used when calculating intervals. |
| Interval | The number of Time Unit values between intervals. For example, 1 second or 1 hour depending on the Time Unit selected. The interval is calculated as the start time plus the interval. Each new interval takes the end time of the previous
interval and create a new interval with that end time as the start time plus the interval. A value is considered to be in
the interval if its time is less than or equal to the start time and less than the end time of the interval.
|
| Hold Interval | The number of Time Unit values to keep the interval before emitting downstream. This value is useful for data that may arrive out of order or late,
it can still be slotted into the correct interval by its time field. This option only helps when a field from the input tuple
is selected as the Timestamp Field. Example: If the current start time for the interval is 01:00:00 and the interval is set to 1 minute the end time is 01:01:00
and if the hold time is set to 5 minutes the emit time will be 01:05:00; if a tuple arrives at 01:03:10 with a 01:00:30 timestamp
it will correctly be added to the 01:00:00-01:01:00 interval but will not be sent downstream until 01:05:00.
|
| Prior Windows | The number of Interval values open prior to the starting interval. For example, if the interval is set to 1 minute and the first input tuple creates
an initial interval from 01:10:00-01:11:00 and prior windows is set to 3, then the system would create three prior windows
such as: 01:09:00-01:10:00 and 01:08:00-01:09:00 and 01:07:00-01:08:00. This allows you to capture some possible late data
during system startup.
|
| Truncate Interval | When enabled, the time value is truncated to the nearest whole value for the time unit selected. For example, if the time give is 01:10:33 and the time unit is minutes, then the time would be truncated to 01:10:00 as the start of the internal. |
| Emit History Size | The number of output tuples to keep in memory to calculate Null Value Expressions as expressed on the Output Tab. Null Value Expressions can use this history to calculate things like the average output value
of a field over the emit history size of outputs.
|
| Log Level | Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE. |
The Missing Data tab is used to check incoming tuples against an expression and if the expression evaluates to true, then replace the field with the given replacement value. This missing data replacement check is done before the input tuple is evaluated further inside the operator. By default, no field values are replaced. The smart fill option can be used to evaluate the current input schema and fill the table with reasonable values.
| Column | Description |
|---|---|
| Field Name | The name of the input field to replace its value if the expression evaluates to true. |
| If Below Is True: | This column is used to enter the expression which is evaluated against the input tuple to determine if the field given in
the first column should be replaced by the value given by the Then Replace with column.
|
| Then Replace with: | This column is the expression for which the results are used to replace the Field Name field of the input tuple before any processing occurs. If the expression is empty a null is used.
|
The Output tab is used to determine what fields are output from this operator. The output can have any number of fields regardless of the number of input fields. Each row of the output fields table is a field to output and the expression column determines what the value of that field will have. The smart fill option can be used to evaluate the current input schema and fill the table with reasonable values.
| Column | Description |
|---|---|
| Field Name | The field name in the output tuple. |
| Expression | The expression which will fill the value of this field in the output tuple. This may be an aggregate expression if the output type is set to aggregate interval. |
| Null Value Expression | Directly before the output tuple is sent downstream each field is checked for null, if it is null then this expression is evaluated and the resulting value replaces the current null value. If this expression is empty no replacement is done and the null value remains. |
| Suppress Row If Null | Directly before the output tuple is sent downstream the value of this rows field is checked for null contents, if the fields
is marked as Suppress Row If Null the entire tuple will not be output and is thrown away.
|
| Selected | If the output type is either Any Selected or All Selected then this column is used to select which fields are used to determine whether the tuple should be output downstream when
a change in value occurs.
|
Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.
Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.
Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.
Caution
Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.
The Data Input port can consist of any schema and is used by the operator's expressions to evaluate and output tuples.