Unpivot

This operator accepts one or more columns along with a list of columns and generates a row for each column specified in the list.

Unpivot operator icon

Information at a Glance

Note: This operator can only be used with TIBCO® Data Virtualization and Apache Spark 3.2 or later.

Parameter

 Description
Category Transform
Data source type TIBCO® Data Virtualization
Send output to other operators Yes
Data processing tool TIBCO® DV, Apache Spark 3.2 or later

The selected columns are removed from the input and are transformed into the following two new columns at the end of the output data set.

  • First column - Values are the names of the selected columns.
  • Second column - Values are the corresponding values in the selected columns.

Input

An input is a single tabular data set.

Configuration

Parameter Description
Notes Notes or helpful information about this operator's parameter settings. When you enter content in the Notes field, a yellow asterisk appears on the operator.
Columns Specify the columns to unpivot. All data types are supported.
Name of Variable Column Specify the name of the first new column. This contains the names of the columns to unpivot.
Note: The value must be alphanumeric. (Regular expression to match are: "^[A-Za-z]+ \\ w*$")
Name of Value Column Specify the name of the second new column. This contains the values of the columns to unpivot.
Note: The value must be alphanumeric. (Regular expression to match are: "^[A-Za-z]+ \\ w*$")
Output Schema Specify the schema for the output table or view.
Output Table Specify the table path and name where the output of the results is generated. By default, this is a unique table name based on your user ID, workflow ID, and operator.
Store Results When set to Yes, the operator saves the results. If set to No, the operator does not save the results.

Output

If you select X columns to unpivot from input with Y columns and N rows, the output data set has (Y-X+2) columns and (X * N) rows.

Visual Output
A table that displays the output of a data set after unpivoting the columns.
Data Output
A tabular data set of the newly created table or view.
Note:
  • The New Variable column contains the names of the unpivoted values in chararray format.
  • The following conditions are applicable for the New Value column:
    • If all columns selected to unpivot are numeric, the resulting value column is double.
    • If all columns selected to unpivot are datetime with the exact same format, the resulting value column is datetime with this same format.
    • For all other cases, the resulting value column is chararray.
  • All null values are kept in the output.

Example

The following example displays the data set created by unpivoting the columns, forming a new table using the Unpivot operator.

Unpivot operator workflow
Data
golf: This data set contains the following information:
  • Multiple columns namely outlook, temperature, wind, humidity, and play.
  • Multiple rows (14 rows).
Parameter Setting
The parameter settings for the golf data set are as follows:

  • Columns: humidity, temperature

  • Name of Variable Column: Variable_Column

  • Name of Value Column: Value_Column

  • Store Results: Yes

Output
The following figure displays the output for the parameter settings for the golf data set.
Unpivot operator Output