Contents
The Spotfire Streaming Adapter for IPFIX allows a StreamBase application to parse IPFIX messages to produce template, option template, and data record tuples.
This section describes the properties you can set for this adapter, using the various tabs of the Properties view in StreamBase Studio.
Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.
Adapter: A read-only field that shows the formal name of the adapter.
Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.
Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.
Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.
Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.
| Property | Description |
|---|---|
| Dictionary File | The dictionary file that specifies that enterprise entities. This file is a JSON formatted file that must conform to the JSON schema described in this section |
| Data Field | The field of the input tuple that contains the binary IPFIX payload. |
| Output Template With Data | If enabled, the template or option template associated with the data records is set in the output tuple. |
| Enabled Pass Through Fields | If enabled, the input tuple is copied to a field in the output tuple. |
| Pass Through Output Field Name | The name of the field to be added to the output tuple which will contain the input tuples data. |
| Capture Transform Strategy | The strategy to use when transforming capture fields for this operator: FLATTEN or NEST. |
| Log Level | Controls the level of verbosity the adapter uses to send notifications to the console. This setting can be higher than the containing application's log level. If set lower, the system log level is used. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE. |
Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.
Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.
Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.
Caution
Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.
The data input port must contain at least one blob field that includes the IPFIX binary information to parse. You must also specify the field name in the Data Field adapter property.
Use the data output port to output parsed IPFIX sets. Sets can contain one or more of Templates, Options Templates, or Data records. The data output port schema is:
| Field Name | Field Type | Description |
|---|---|---|
| Header | tuple |
The parsed packet header with the following fields:
|
| Sets | list(tuple) |
A Set is a collection of records that have a similar structure, prefixed by a
header. In an IPFIX Message, zero or more Sets follow the Message Header.
There are three different types of Sets: Template Sets, Options Template
Sets, and Data Sets. The sets list tuple contains the following fields:
|
The dictionary output port will output the parsed dictionary in tuple form, for reference.
The schema for the dictionary output port is:
| Field Name | Field Type | Description |
|---|---|---|
| Enterprises | list(tuple) |
A list of all the enterprises in the dictionary with the following schema:
|
The status output port will output tuples giving relevant information.
The schema for the status output port is:
| Field Name | Field Type | Description |
|---|---|---|
| Status | String | A string describing the current status of the adapter. |
| Time | timestamp | The timestamp that the status occurred. |
| Info | list(tuple) | A list of name value pairs of data to got more context to the status. |
The following is a list of status outputs and the associated objects:
| Status | Info Elements |
|---|---|
| Parse Error |
|
| Field Parse Error |
|
| Missing Dictionary Field |
|
The dictionary is used to determine what fields are parsed from the IPFIX packets.
The following is the JSON schema required for the dictionary file.
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"includeDictionary": {
"type": "array",
"items": {
"type": "object",
"properties": {
"file": {
"type": "string"
}
},
"required": [
"file"
]
}
},
"enterprises": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"organization": {
"type": "string"
},
"contact": {
"type": "string"
},
"email": {
"type": "string"
},
"entities": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"name": {
"type": "string"
},
"dataType": {
"type": "string"
},
"dataTypeSemantics": {
"type": "string"
},
"units": {
"type": "string"
}
},
"required": [
"id",
"name",
"dataType"
]
}
}
},
"required": [
"id"
]
}
}
},
"required": [
"enterprises"
]
}
The following sections list the major categories of the dictionary file and provide an overview of their usage.
The includeDictionary section is an array of files that allows one dictionary file to include another dictionary file. This enables you to organize your dictionaries and combine them into a single dictionary.
-
file—The relative path to the linked dictionary file to include.
The enterprises section allows you to add enterprise values.
-
id—A long value representing the enterprise.
organization—(Optional) The organization name.
contact—(Optional) The enterprise contact.
email—(Optional) The enterprise contact email.
entities—An array of entities associated with this enterprise:
-
id—The integer item value representing the item. This value is left 0 padded and used to match the packets item type.
-
name—(Optional) The description of this item.
-
dataType—The data type of this entity which is used to determine the associated StreamBase data type when creating the output schema, this value is also used to determine how to parse the entity field from the binary data. Allowed values are:
-
boolean—Maps to a StreamBase boolean field
-
dateTimeMicroseconds—Maps to a StreamBase tuple containing a
DateTimeSecondstimestamp field and aFractionslong field -
dateTimeMilliseconds—Maps to a StreamBase timestamp field
-
dateTimeNanoseconds—Maps to a StreamBase tuple containing a
DateTimeSecondstimestamp field and aFractionslong field -
dateTimeSeconds—Maps to a StreamBase timestamp field
-
float32—Maps to a StreamBase double field
-
float64—Maps to a StreamBase double field
-
ipv4Address—Maps to a StreamBase string field
-
ipv6Address—Maps to a StreamBase string field
-
macAddress—Maps to a StreamBase string field
-
octetArray—Maps to a StreamBase blob field
-
signed16—Maps to a StreamBase int field
-
signed32—Maps to a StreamBase int field
-
signed64—Maps to a StreamBase long field
-
signed8—Maps to a StreamBase int field
-
unsigned16—Maps to a StreamBase int field
-
unsigned32—Maps to a StreamBase long field
-
unsigned64—Maps to a StreamBase long field
Note
Java does not fully support unsigned data types, and as such StreamBase may display any unsigned64 with a length of 8 as a negative value.
-
unsigned8—Maps to a StreamBase int field
-
-
dataTypeSemantics—The data type semantics associated with this entity. This value is not used internally and is only output on the dictionary output port for reference.
-
units—The units associated with this entity. This value is not used internally and is only output on the dictionary output port for reference.
-