Tips on Introspecting Large Data Sources
Large data sources with complex schemas and big tables with lots of columns have a correspondingly large amount of metadata that can pose challenges for TDV. For TDV, it is not the amount of data stored, but rather the complexity and amount of metadata that defines the data source that is the most important factor for performance, particularly during introspection.
Introspection can be broken down into three phases.
Understanding these phases can improve introspection performance for large data sources:
Each of these phases has independent performance considerations.