In-database and in-memory data loading in the TIBCO Cloud™ Spotfire® web client

When you analyze data from external data sources, you can either import data and analyze it within the Spotfire data engine (in-memory) or you can keep the data in the external system (in-database).

Configuration of the load method of a connection is done in an installed client, and this will determine how Spotfire should load the data upon opening the connection. In the web client, you cannot select the load method when working with data connections.

Note: To know the load method used for a data table in an analysis, select the table of interest and inspect the data connection node in the Data canvas.

In the TIBCO Cloud™ Spotfire® web client, to ensure stable performance, certain limits exist on data loading from external systems. Depending on the load method used, keep these limitations in mind when working with data connections in the web client.

Table 1. In-memory loading limitations
Limitation	Description
Data import row limit	The maximum number of rows in a data table that can be imported from an external data source in TIBCO Cloud Spotfire web client is 20 million. If an imported data table contains more rows, only the first 20 million rows will be included. If you want to work with data that contains more rows, this is supported by configuring the connection to allow in-database loading of the data tables. This configuration can be done in an installed client, by the connection author.

Table 2. In-database loading limitations
Limitation	Description
External data row limit	The maximum number of rows that can be retrieved and visualized is 500,000, when working with in-database data. This does not limit the number of rows allowed in the underlying data table in the external data source, but only the amount of data that can be fetched in a query to the data source. The limit affects, for example, visualizations, the expanded data panel, and certain filter types. You might exceed this limit if you configure a complex visualization where too many items are shown on the axes, or if you create a filter for a column that contains too many unique values. You can often work around the limit by, for example, filtering the data or configuring the visualization to show fewer items. If a visualization or filter is configured to show more than 500,000 items, these elements will not be rendered in the analysis, and an error message will be displayed.
Table visualization row limit	The table visualization can display a maximum of 10,000 rows of data, when working with in-database data. If the data table contains more rows, only the first 10,000 will be visualized.