Column Filter
This operator selects a subset of the columns from the data source. Only the columns selected remain in the output data set.
Information at a Glance
Note: This operator can only be used with TIBCO® Data Virtualization and Apache Spark 3.2 or later.
|
Parameter |
Description |
|---|---|
| Category | Transform |
| Data source type | TIBCO® Data Virtualization |
| Send output to other operators | Yes |
| Data processing tool | TIBCO® DV, Apache Spark 3.2 or later |
This operator is particularly helpful when a data source has several columns that are not needed for the data analysis workflow.
Input
An input is a single tabular data set.
Configuration
| Parameter | Description |
|---|---|
| Notes | Notes or helpful information about this operator's parameter settings. When you enter content in the Notes field, a yellow asterisk appears on the operator. |
| Columns | Specify the columns to be made available for analysis.
Click Select Columns to open the dialog to specify columns. See the Columns dialog for more information. |
| Output Schema | Specify the schema for the output table or view. |
| Output Table | Specify the table path and name where the output of the results is generated. By default, this is a unique table name based on your user ID, workflow ID, and operator. |
| Store Results | When set to Yes, the operator saves the results. If set to No, the operator does not save the results. |
Output
Visual Output
A table that displays the output of a data set.
Data Output
A tabular data set of the newly created table or a view.
Example
The following example shows the data set created from data fields of the given data set, forming a new table using the Column Filter operator.
Data
golf: This data set contains the following information:
- Multiple columns namely outlook, temperature, wind, humidity, and play.
- Multiple rows (14 rows).
Parameter Setting
The parameter settings for the golf data set are as follows:
-
Columns: outlook, temperature, wind
-
Store Results: Yes
Output
The following figure displays the output for the parameter settings for the golf data set.
