Types of Errors During Big Data Import

You can resolve the errors found in the data source file. Some common scenarios are as follows:
  • If the number of data source attributes does not match with the number of header attributes in the input file, the attributes mismatch error is displayed
  • The errors for the attribute which has an error, such as an incorrect date format or the string value exceeded the maximum characters
  • Classification errors against each record
  • Unexpected errors against each record
  • Duplicate data errors
  • Invalid repository attribute errors
  • Invalid multivalue attributes
  • Invalid relationship attribute

If Apache Spark and Apache Hadoop are not properly installed or configured, the SPARK-20005 Fatal error is displayed. For resolution of installation and configuration related errors, see the "Big Data Import Errors" section in TIBCO MDM System Administration.

After restarting the Apache Spark master and worker nodes if you initiate the big data import, the import fails. In this case, instead of restarting the TIBCO MDM server, you must clear the spark context by using the Spark MBean. For information on using the Spark MBean and performing the clear spark context operation, see the "TIBCO MDM Monitoring and Management Using JMX chapter" in TIBCO MDM System Administration.