Push-compute data

If you have the Spotfire Industry Pro license, you might be able to select the Push compute option when you add data using certain connectors. This option will keep the data in the database in the same way as external data, but it also makes it possible to perform transformations and data function calculations using the platform’s data engine.

See also Load methods for an overview of the different ways you can work with data in Spotfire.

The push compute option lets you do more things with your external data, such as adding rows or columns, and transforming the data entirely on the data platform side. Data functions can be run using the external data engine, provided that the inputs and outputs also are handled in the external system. See Using a specific calculation engine for data functions, for more information about push-compute data functions. Additionally, transformations such as join, pivot, and so on, are possible to use with the supported platforms and you can easily move data between Spotfire in-memory and the external system.

With push compute, you can upload additional data from Spotfire to a temporary clone in the data platform and perform all computations there, using the data platform’s data engine.

Note: It is important to use the same data connection for all activities that you want to perform within an analysis, and not create new ones.

The key difference between the Push compute and External (in-db) data connection options is that, with push compute, Spotfire manages each user’s work as temporary data tables residing on the target data platform as "engine states". This allows transformations and data functions to execute as if they were working on in-memory data tables.

Therefore, a database on the data platform must be reserved for and managed by Spotfire. During the course of the execution, Spotfire creates temporary schemas and tables, and cleans them up after the user has closed the data connection.

Other things that might be important to consider when using push compute can be the performance and/or data protection issues, if your data platform resides in a different country than where you are operating. Always discuss with your data administrator or management if you know you are handling sensitive or protected data.

Preparation

Before the push compute functionality can be used, there must be some preparations done by both end users and administrators:

You have access to a Snowflake or Databricks data platform.
A database where engine states are stored must be available on the data platform.
The required roles must be created and configured on the data platform by the system administrator. See the separate topics listed below for each system.
For Databricks connections, Unity Catalog must be enabled.

Preparation for connecting to Snowflake (by end users)

Follow the Snowflake instructions to install the Snowflake ODBC driver for Windows.

When creating the Snowflake connection data source you must select the Allow the load method 'Push compute' check box to enable the Push compute option in the data connections.

Preparation for connecting to Databricks (by end users)

Install the Databricks ODBC driver.

When creating the Databricks connection data source you must select the Allow the load method 'Push compute' check box to enable the Push compute option in the data connections, and select a compute cluster from the drop-down list.