Store Intermediate Results
Many operators can be configured to store intermediate results in a database table or view, or in a Hadoop file.
- Database Operators
-
For database operators, the user can select whether to create intermediate results as a table or view, and where the results are created. The default values for the results locations use workflow variables to enable the user to make changes across the entire workflow without having to edit each individual operator.
- Hadoop Operators
- For Hadoop operators, the user is often able to choose whether to store intermediate results. The default values for the results location use workflow variables to enable the user to make changes across the entire workflow without having to edit each individual operator. The directory is based on the value of the
@default_tempdir workflow variable.
In certain cases, an operator can only be connected to another operator if its Store Results? parameter is set to True. This is because certain operators (for example, most modeling operators) must work on files rather than an intermediate result set. Team Studio automatically sets Store Results? to be True when this is required.