When to Use Database Loader

Database Loader provides fast loading of data with relationships and should be your preferred option if you need to import large initial data in a single load with or without relationships.

Database Loader is faster because it does not include:

  • Workflows
  • Approval
  • Validations
  • Record history
  • Named version

Use Database Loader when the imported data is clean, does not require any validation, and only for initial versions. Data is mapped from data sources to input maps and imported as is.

Database Loader performance depends on the following factors:

  • Database setup: database Loader performs bulk operations (mostly inserts, some updates and deletes) therefore the larger the batch size, the more database resources it requires. If you cannot change the database parameters, consider using smaller sets per import.
  • Number and size of attributes: more attributes and larger attribute sizes take longer to import.
  • Indexes on MCT tables: although you can create many indexes on MCT tables to facilitate searches, this slows down inserts and deletes. Determine if you really need these indexes and consider if you can drop the indexes during the initial load.
  • Number of records in a repository: a larger size means the database needs to work harder to insert and delete the records. To address this issue, ensure that the database advisors do not report any problems with file systems, segments, or table spaces. You can also partition your data.
  • Partitioning strategy: verify how your indexes are partitioned and that your partitioning is appropriate and not causing slow inserts.

You cannot improve Database Loader performance by adding CPU or memory to TIBCO MDM or cache instances.