Variable

This operator is used to define variables created from data fields of the input data set, forming a new table or view.

Variable operator icon

Information at a Glance

Note: This operator can only be used with TIBCO® Data Virtualization and Apache Spark 3.2 or later.

Parameter

 Description
Category Transform
Data source type TIBCO® Data Virtualization
Send output to other operators Yes
Data processing tool TIBCO® DV, Apache Spark 3.2 or later
Important: The created variables are static in nature. They cannot dynamically change during runtime.

You can also use the Variable operator to divide the data rows into quantiles, adding quantile variables to the data. Dividing the data into smaller and smaller divisions (quantiles) provides an understanding of the overall data distribution patterns.

Note: Column names in the script mode must be enclosed by a backtick (` `) if the column name contains special characters or multi-byte characters such as Chinese, Japanese, or Korean characters. Do not use double quotes to enclose the column names.

Input

An input is a single tabular data set.

Bad or Missing Values
Null values are not allowed and result in an error.

Configuration

Parameter Description
Notes Notes or helpful information about this operator's parameter settings. When you enter content in the Notes field, a yellow asterisk appears on the operator.
Variables Specify the expression(s) to create the new Variable column(s). Click Define Variables to define the variables. For more information, see Define Variables dialog.
Note: The SQL expression must adhere to the SQL syntax supported by Spark since this is Spark SQL. For information on the syntax, see Spark SQL Syntax and Expressions.
Quantile Variables If the new variable to create is a quantile variable, select the required column(s) to use for deriving the quantiles.

The possible quantile types are Average Ascend (which automatically creates the bins) and Customize (which manually defines the variable bins).

Note: Automated variable binning can run on only databases that support NTILE, such as Greenplum, PostgreSQL, Oracle, and SQL Server. It is not supported on databases such as Teradata, MySQL, and MariaDB that do not currently support NTILE.

For more information, see Define Quantile Variables dialog.

Columns Specify the columns to include. For more information, see Select Columns dialog.
Output Schema Specify the schema for the output table or view.
Output Table Specify the table path and name where the output of the results is generated. By default, this is a unique table name based on your user ID, workflow ID, and operator.
Store Results When set to Yes, the operator saves the results. If set to No, the operator does not save the results.

Output

Visual Output
A table that displays the output of a data set. To see all of the data rows in addition to the derived variables, select all columns for the Columns parameter.
Data Output
A tabular data set of the newly created table or view.

Example

The following example displays the variables created from data fields of the given data set, forming a new table using the Variable operator.

Variable operator Workflow
Data
golf: This data set contains the following information:
  • Multiple columns namely outlook, temperature, wind, humidity, and play.
  • Multiple rows (14 rows).
Parameter Setting
The parameter settings for the golf data set are as follows:

  • Variables: test

  • Columns: temperature, humidity, wind

  • Store Results: Yes

Output
The following figure displays the output for the parameter settings for the golf data set.
Variable operator Output