Database Loader Usage

Use Database Loader when data being imported does not need any validation or cleansing. Data is mapped from data sources to input maps, and imported as-is. Database Loader consists of the following two modes:

  • Load Records: Use to load record data with a relationship.
  • Load Relationship: Use to load a relationship between existing records.

Each mode includes the following two options:

  • Fresh Data: The Fresh Data option requires clean data and only initial version records. Initial version records signify that the same record must not already exist in the repository and not even in the deleted state. Clean data signifies that duplicate and deleted records do not exist in data source.
Note: If the data is not clean or it is not initial version records, an error occurs and you need to run the rollback script.

Consider that the following record hierarchy is available in TIBCO MDM:
CUSTOMER1 > BANK1 and BANK1 > LOCATION1 and BANK1 > LOCATION2,
Input maps are created to support this hierarchy and data source.

To load this hierarchy, a typical data source can be:

  • Example 1

    CUSTOMER1, BANK1, LOCATION1

    CUSTOMER1, BANK1, LOCATION2

    In this example, CUSTOMER1 and BANK1 records are duplicated. Hence, this data source cannot be used for the Fresh Data option.

  • Example 2

    CUSTOMER1, BANK1, LOCATION1, LOCATION2

    This example contains clean data, and can be used for the Fresh Data option.

  • Record Versioning: Record Versioning option is similar to the regular Direct Load import with some limitations. Using this option, you can perform the following actions:
    • Import initial version records
    • Modify records by incrementing record versions: In this case, older version records are overwritten by new versions, whereas if previous relationships exist; they are merged. If an older version is already in workflow, then the processing of such records is skipped by Database Loader. An entry is created in the skipped records log file, which can be downloaded from the Event Log screen.
    • Duplicate data source records: In this case, only one record is processed. Other records are ignored, although relationships from all records is processed.
Note: Fresh Data mode is faster than Record Versioning mode.

The mode in which Database Loader should run is determined by the input map’s Approval Option, Mode, and the Fresh Data check box.

If you want to import deleted records, clear the Fresh Data check box.

Note: The Database Loader is enhanced for faster and better tracking of data loading progress, error reporting file with summary of the errors. The enhancement works only for initial load having Fresh Data enabled, the new mode can be enabled by disabling bulk mode. The bulk mode is disabled by setting the com.tibco.cim.dbloader loadrecord.bulkmode to false and the com.tibco.cim.dbloader.loadrelationship.bulkmode to false. The new mode is recommended for initial load cases.