Workflow Variables

It is possible to override the default workflow parameters using workflow variables.

This practice fine-tunes the parameters for only the specified workflow.
The value of the workflow variable is substituted appropriately at run time, which provides more flexibility at design time.
To view a workflow's variables, on the Actions drop-down list box, click Workflow Variables.

Default Workflow Variables

Every workflow contains default variables, and each variable begins with the @ symbol:

@default_schema: used as the default schema for database output
@default_tempdir: used as the default directory for intermediate Hadoop files
@default_prefix: used as the default prefix for output tables and output files
@default_delimiter: used as the default data delimiter for CSV and Hadoop files.

Note: The default variables cannot be deleted, but their values can be changed to override the global defaults at the workflow level.

Spark Workflow Variables

We do not support setting Spark variables on a workflow-wide level. It is a Spark best practice to optimize the Spark parameters per job or operator that you want to run. To make this process easier, we created Spark Autotuning to help you choose the most performant values for your Spark job settings. See also Spark Optimization For Data Scientists for more information.

Defining New Workflow Variables
In addition to the default variables that every workflow contains, you can also create new workflow variables.
Overriding Hadoop Data Source Parameters Using Workflow Variables
It is possible to override the default Hadoop data source parameters using the workflow variable settings.
Team Studio Operator Job Names
The following is a list of Team Studio job names that can be referenced for overriding specific Hadoop MapReduce data source parameters only for particular operator tasks.