User Guide > Retrieving Data Source Metadata > Tips on Introspecting Large Data Sources
 
Tips on Introspecting Large Data Sources
Large data sources with complex schemas and big tables with lots of columns have a correspondingly large amount of metadata that can pose challenges for TDV. For TDV, it is not the amount of data stored, but rather the complexity and amount of metadata that defines the data source that is the most important factor for performance, particularly during introspection.
Introspection can be broken down into three phases.
Understanding the Resource ID Fetching Phase
Understanding the Introspection Plan Creation Phase
Understanding the Introspection Plan Implementation Phase
Understanding these phases can improve introspection performance for large data sources:
Tips to Improve Performance of Resource ID Fetching
Tips to Improve Performance of Introspection Implementation
Tips for Reintrospecting Large Data Sets
Each of these phases has independent performance considerations.