So that TDV has complete metadata about all of the resources in your data set, do at least one full data source scan to retrieve all possible object names (resource identifiers) from within the data source. The initial fetch loads the resource list with basic metadata for all catalogs, schemas, tables, and procedures.

This phase corresponds to Invoking Data Source Introspection. After you have invoked introspection, TDV begins populating the resource list in the Data Source Introspection dialog with the resources it detects.

When obtaining this resource ID list, keep these things in mind:

The actual fetching of Resource ID from data sources is multithreaded to provide concurrency. One thread per container (data source root, schema, catalog, or other type of container) is created. The degree of parallelism (maximum number of threads) is either the number of processors, or half of the data source maximum connection pool size, whichever is less. Degree cannot be less than 1.

After the resource IDs are fetched and placed within a cache in the TDV, the resource IDs are directly fetched from the cache, unless the cache has been invalidated and needs to be recreated. This cache-based fetch is single-threaded but fast. The results of the fetch are streamed to Studio.

The resource identifiers are invalidated and the resource ID list is recreated when:

Reintrospecting existing resources does not recreate the cache unless the user has requested that new resources be detected for adding.