This release contains the following enhancements:
The Ensure group management parameter was added to the Manage group workflow service. When this option is enabled, the workflow's Accept option only displays if the current group is resolved and has no suspect records.
The Back to the main view button was re-labeled to Review the merged group to more closely describe the button's behavior.
The Full text algorithm was updated to improve the consistency of matching operation results.
When merging records or managing groups, a new vertical display option is available. This option displays the records in columns instead of rows. Administrators can set the default view with the new ebx.addon.mame.manageGroup.record.display property in the ebx.properties file. Use the values vertically or horizontally to define the view orientation. Please note that the vertical display option is not available when merging business objects.
The UI for viewing pre-processing information was improved as follows:
Now, it only displays a field configured for matching one time, even if it is used in multiple decision tree nodes.
A value column was added to display the values being compared.
The fields are grouped by their tables.
The UI for the decision tree result explanation was updated to improve readability.
It is now possible to transfer configuration information for data models located inside a registered module via the EBX® staging feature.
The foreign key alignment phase was added to the progress bar when running a matching operation.
A warning is now displayed if an algorithm that is incompatible with the defined search strategy is selected in the decision tree. This warning no longer displays when evaluating matching results.
This release contains the following performance, technical, and API enhancements:
The pre-processing phase now takes into consideration the decision tree content to reduce the cluster size, which can lead to improved performance.
During the pre-processing phase the balancing of matching field scores was improved so that each field equally affects the final similarity score.
When simulating a match operation using the API, a procedure is no longer opened every time. This prevents a lock of the dataspace and any negative effects on TableTrigger.
The separate procedures for matching, grouping, and merging were merged into a single procedure.
Performance was improved when running a match operation with a selection of records.
The REST API to accept or reject suspect records is now available.
It is now possible to create a primary key generator to ensure that specific values from a composite primary key are survived into a golden record.
As the add-on UI is under development, some images in the documentation might not exactly match those in the UI.
This release contains no changes in functionality.
Take into account the following when upgrading from TIBCO EBX® Match and Merge Add-on version 3.0.0:
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
If upgrading from any version prior to 6.2.0, a performance improvement was made to better balance the weights of fields that participate in matching. The automatic balancing occurs during the pre-processing phase. This helps mitigate issues that might occur when fields include many values. As a result of this update, matching operation results might be impacted. To mitigate any negative impacts, you can test and adapt your existing weights to achieve the desired matching outcome. Alternatively, you can set the ebx.clustering.balance property to false in the ebx.properties file to restore the legacy behavior.
This release contains no changes to third-party libraries.
This release contains the following closed issues:
[MAME-8315] The record's state and relationship are not rolled back when cancelling a matching operation after the exclusion phase.
[MAME-8826] A warning message is not included in the validation report when the default search strategy is used with the fuzzy algorithm.
[MAME-8828] A warning message is not included in the validation report when the phonetic search strategy is used with the distance algorithm.
[MAME-9658] Locked fields are not fixed in place when scrolling in the Merge screen.
This release contains the following known issues:
[MAME-8084] The Evaluate matching service returns inconsistent results for related matching fields that have the DateTime data type.
[MAME-8857] It is not possible to accept or reject suspects in groups that contain a large number of records.
Foreign key alignment of recursive foreign keys is not possible.
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
[MAME-9049] A manual merge operation cannot be performed on more than 400 records.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
This release contains the following updates focused on performance improvements:
Foreign Key alignment during the merging phase was optimized.
Instead of relying on snapshots, a new method is used to check the differences between transactions.
The API was updated to improve performance when getting lists of suspects or records contained in a group.
This release contains the following enhancements:
New rules are applied to survivor record selection for the instances where no records, or multiple records satisfy required conditions.
When running the Purge old results service, out-of-date matching selections are now also purged.
Foreign key alignment of composite primary keys during automatic merge was improved.
The relationship between selected records is now removed for any suspect record that is accepted as a match when managing groups.
The readability of information written to the logs regarding foreign key alignment was improved.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains the following changes to functionality and behavior.
If a field configured to participate in matching meets either of the following criteria, it is automatically disabled during the pre-processing phase to improve performance:
The field's type is boolean.
The entire column is null.
Special characters are now preserved when using the add-on's distance matching algorithms.
The Spring framework was updated to version 5.3.29.
This release contains the following closed issues:
[MAME-6368] The Merge and Purge old results legacy services still display on the Merged Record Association screen.
[MAME-8332] Field values that satisfy the merge policy requirements are still preselected when the survivor record is read-only.
[MAME-8656] A NullPointerException occurs in log4J2 during Liberty startup.
[MAME-9352] The incorrect survivor field value is used when multiple records satisfy merge function conditions.
[MAME-9383] When auto-merging, do not select the smallest primary key to be the survivor record when no records satisfy the Record selection function.
[MAME-9418] The inline matching is marked as Deprecated in the breadcrumb when running the service.
[MAME-9420] For Auto-merge, result of aligning some foreign and primary keys are incorrect if the keys are set to read-only on dataset permissions.
[MAME-9421] For Auto-merge, alignment of foreign and primary keys is incorrect when a related table has a primary or foreign key, and auto-incremented primary key, and one of those keys is read-only by dataset permission.
[MAME-9422] For Auto-merge, alignment of foreign keys does not work for all related tables if one FK field from any related tables is set to Read-only.
[MAME-9423] For Auto-merge, alignment of foreign and primary keys is incorrect when a related table has a primary or foreign key, and auto-incremented primary key, and one of those keys is read-only by Access rule.
[MAME-9466] The align foreign key function (in another dataspace) does not work correctly when the merge service runs in a child of the referenced dataspace configured on the data model.
[MAME-9505] Medium and large numbers of merged records cannot be modified (changed to <unset>).
[MAME-9519] The selected auto-Golden records in Related tables incorrect after merged several auto-created records by selecting each record.
[MAME-9536] The result column in the decision tree explanation popup disappears when using an evaluation method which other than Average.
[MAME-9539] A blank Merge related screen when creating a new golden on a business object.
This release contains the following known issues:
[MAME-8084] The Evaluate matching service returns inconsistent results for related matching fields that have the DateTime data type.
[MAME-8857] It is not possible to accept or reject suspects in groups that contain a large number of records.
Foreign key alignment of recursive foreign keys is not possible.
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
[MAME-9049] A manual merge operation cannot be performed on more than 400 records.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
This release contains the following new features:
It is now possible to use the replication feature on Business objects.
Data comparison nodes now have the option to use operators or algorithms to determine how to values are compared.
The following user experience enhancements were made:
The display order of fields in merge policies was updated so that Auto create new golden is now at the top. This is more intuitive as this value can impact the availability of other options.
The explanation for the evaluation of matching execution now includes additional parameters to better explain the results.
Performance was improved when changing the state of selected records to [Unset].
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes to functionality and behavior.
The FasterXML jackson-databind library was updated to version 2.15.2.
This release contains the following closed issues:
[MAME-8983] [JAVA API] An exception is thrown when no duplicates are found in a group.
[MAME-9210] All records display in the Merged association tab when opening a record that is not golden.
[MAME-9289] A different method should be used to detect whether a table belongs to a Business object, or individual table configuration.
[MAME-9311] After being deleted in the Matching fields tab, fields in a data comparison node cannot be re-created that use a range data type (Date, Time, Date and time, integer, decimal).
[MAME-9358] When using staging to import the matching configuration, it fails if the decision tree in the source environment uses Predicate.
[MAME-9390] The decision tree result is incorrect when: 2 or more matching fields are used, one field is from a related table, and it's values are null due to being excluded by the Exclude related records where setting.
This release contains the following known issues:
Foreign key alignment of recursive foreign keys is not possible.
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
[MAME-9049] A manual merge operation cannot be performed on more than 400 records.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
This release contains the following new features:
The add-on was adapted to ensure compatibility with the EBX® staging feature.
A page loading indicator now displays when performing a manual merge.
When configuring inline matching for a workflow, it is now possible to use the Matching policy parameter to specify a matching policy other than the default policy configured for the table. The default policy is used when this parameter is undefined.
All snapshots related to the add-on now include a MAME_ prefix.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains the following changes to functionality and behavior:
The Search before create service no longer needs to be enabled by an administrator. Additionally, when running the service, the matching policy is selectable. The chosen matching policy will determine which fields are available to search and the criteria used to locate potential matches. The Search before create service is still available from the matching policy configuration, however it is marked as deprecated.
When a matching table configuration is deleted, now all related replications are also removed.
The following default options were changed in comparison node configurations:
The default value for Comparison function is now All fields match.
For the Full text, Fuzzy, and Hybrid fuzzy algorithms, the Minimum score now defaults to a value of 100.
The Consolidated view was deprecated. It is still available to access, but it is planned to be removed in a future release.
The Spring framework was updated to version 5.2.24.
This release contains the following closed issues:
[MAME-5690] The screen does not display correctly in a data workflow when the matching field of the default matching policy is deleted.
[MAME-6629] Color properties are not applied correctly.
[MAME-8088] The tooltip for 'Enable Matching' is not localized in the Administration pane.
[MAME-8104] Records that include the '+' character in the primary key cannot be viewed when accessing their group.
[MAME-8807] Some records are incorrectly grouped after running a matching operation.
[MAME-8848] The undo option is still clickable for values where it should not be available.
[MAME-8864] The Manage group screen does not display correctly after changing the view.
[MAME-8931] Records' false positive icons are not removed in the Manage group screen after deleting the records from the metadata service.
[MAME-8934] The Search before create service cannot be executed when the Matching fields tab includes invalid matching fields.
[MAME-8935] In workflows, the Search before create screen cannot be re-accessed after performing an inline merge and updating a record in list of potential duplicates.
[MAME-9016] The add-on's manage group feature cannot be accessed when primary keys include a foreign key.
[MAME-9038] Redundant rows can display in the Manage group screen.
[MAME-9059] A foreign key field can be configured to use other algorithms besides the Exact algorithm.
[MAME-9061] The error message from previous phase is not cleared after changing field.
[MAME-9205] Nothing happens when clicking the Preview button on primary key or foreign key field in the Lineage screen.
[MAME-9206] Display ALL services under Actions menu in the association of Merged records.
[MAME-9225] The screen does not display correctly after previewing a record in the Manage group screen.
[MAME-9248] Matching results are provided using the light relationship instead of the strong relationship when both are configured on a business object.
This release contains the following known issues:
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
[MAME-9049] A manual merge operation cannot be performed on more than 400 records.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: March 2023
This release contains the following new features:
The add-on was adapted to ensure compatibility with the way EBX® now handles URL encoding. See the EBX® release notes for additional information.
Group management:
The Manage group screen user interface and user experience were updated and improved.
Any existing custom table views can now be applied in the Manage group screen.
All group management functions are now available when accessing this feature through a workflow. Additionally, the Close button is now available on this user service in workflows.
Manual merge:
The user interface and user experience were updated and improved in this release for the Merge view. The view includes a new merge progress tracker that includes sub-steps when merging a business object's child tables.
A new Undo button was added to the Merge view. This button allows you to revert the most recent values selected for a golden record. The reverted value is highlighted in the UI.
The merge Summary screen was updated to display successfully merged records along with the icons to access the record's detailed view.
Hybrid fuzzy algorithm:
When using the hybrid fuzzy algorithm in a decision tree node, you can now customize the values used by the algorithm for: Synonyms, Levenshtein 1, and Levenshtein 2.
The Levenshtein transformation logic was updated to ensure that short strings return more accurate results.
The No fields match option was removed from decision tree comparison nodes. If you used this option in one of your add-on configurations, navigate to: Administration > TIBCO EBX® Match and Merge Add-on > Table activation and settings. The Status column will alert you to any configurations where you must select a different comparison function.
The option to automatically trigger matching operations is now available for business objects. Only the After submission option is available to use with business objects, either On creation or On update.
Evaluation of matching:
When evaluating matching results and viewing results for a data comparison node, the displayed information now includes the table that each compared field belongs to.
The similarity results were updated to include additional information of why records do not complete the pre-processing stage.
New Java APIs were introduced to:
Evaluate matching results.
Retrieve the trusted sources list.
Perform multiple merges in one transaction.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
The behavior of the Modify merged records option was reverted. You can once again choose what the add-on does when users modify merged records. An additional option was also included that is a combination of Keep records in group and Change to <unset>.
The SnakeYAML library was updated to version 2.0.
This release contains the following closed issues:
[MAME-7611] An inconsistent matching result is shown in the diagram when evaluating matching results.
[MAME-8039] An error displays after running the Evaluate matching service multiple times.
[MAME-8114] Matching cannot be evaluated when using a group's field and a related field as matching fields.
[MAME-8174] When managing relationships during a manual merge, it takes a long time to display records.
[MAME-8230] [BO] There is no new golden record but the Apply merge policy button is still enabled.
[MAME-8235] An incorrect error message displays when manually merging and the main table's primary key is not auto-incremented.
[MAME-8310] The record order in when managing a group in the Consolidated view is incorrect.
[MAME-8320] An incorrect tool-tip displays after modifying values in a golden record.
[MAME-8321] The matching field is changed when duplicating a matching policy.
[MAME-8337] Hidden field values are not updated after the merge process finishes.
[MAME-8340] The warning banner does not display when accessing a group that includes at least one hidden record.
[MAME-8342] Records are not sorted correctly when accessing the Merge view when managing a group.
[MAME-8348] [UI] When hovering over the False positive icon in the Manage group screen, the cursor displays as a hand.
[MAME-8389] A false positive relationship is not created after merging records in the inline screen.
[MAME-8413] The State is duplicated the Identified as false positives tab.
[MAME-8439] Incorrect matching results are returned when using light relationships (FK from matching table to target table OR over a link table).
[MAME-8441] The back button does not work as expected after completing a merge.
[MAME-8539] Foreign keys do not display correctly in a business object's Manage group view.
[MAME-8542] Some text is not translated into French.
[MAME-8669] All add-on services still display in the Actions menu after pausing the corresponding matching configuration.
[MAME-8677] Matching cannot be evaluated for selected records when using a matching policy without having a merge policy defined in the related table.
[MAME-8844] The decision tree result is inconsistent when evaluating matching for a URI field through a relationship.
[MAME-8877] An error displays after unmerging an auto-created golden record that had been merged multiple times.
[MAME-8878] Unexpected information is included in the log file.
This release contains the following known issues:
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: February 2023
This release was updated to ensure compatibility with TIBCO EBX® Add-ons Bundle version 5.6.1.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes in functionality.
This release contains no changes to third-party libraries.
This release includes no closed issues.
This release contains the following known issues:
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: November 2022
This release contains the following new features:
The UI was updated for manual merge operations on the tables that comprise business objects.
The stewardship UI was updated so that lineage and metadata can be viewed for a business object's related tables.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes in functionality.
The FasterXML jackson-databind library was updated to version 2.13.4.2.
This release includes the following closed issues:
[MAME-7953] The manual merge view is slow to load if the table has a merged records association.
[MAME-8157] The Align foreign keys step does not display when a merge policy attribute is defined and Use for manual merge is set to No.
[MAME-8201] Hidden field values are still highlighted when selecting a survivor field by merge function.
[MAME-8265] Matching state is cascaded to child levels when unmerging even though the child records are excluded.
[MAME-8283] When using matching fields from related tables with the Exact algorithm and Both value are null = Match, there are inconsistent matching result between Evaluate Matching and See details.
[MAME-8286] An HTML block is displayed instead rendering the Decision tree and Business object activation.
[MAME-8318] Deleted records display in the Data selection view of the Supersede survivorship service.
[MAME-8326] The Manage group screen should be reloaded after a successful manual merge operation.
This release contains the following known issues:
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: September 2022
This release contains the following new features:
Matching and merge operations can be configured on multiple tables that comprise EBX® business objects.
The Manage group view is able to display records from multiple tables in a business object.
To improve the user experience, the progress bar was updated.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes in functionality.
The Apache Taglibs library was removed from this release.
This release includes the following closed issues:
[MAME-7661] The content does not display correctly when there are more than 50 errors in a validation report on the Consolidated view.
[MAME-7844] There is a disparity between matching results when using Evaluate matching vs Run match.
[MAME-7874] After rejecting a suspect record, users are navigated away from the Manage group screen and back to the tabular view.
[MAME-7989]DecisionTreeComparator is not thread-safe.
This release contains the following known issues:
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: July2022
This release contains no new features.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes in functionality.
This release contains no changes to third-party libraries.
[MAME-7715] The merge process is slow when changing the record state to Unset.
This release contains the following known issues:
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: June 2022
This release contains the following new features:
The following algorithms were added:
Phonetic full text: An algorithm best used for strings. It can recognize the phonetic equivalent of two words spelled differently using the Beider-morse phonetic tokens. The comparator takes into account the synonyms and stop words defined in the data model.
Hybrid fuzzy: An algorithm best used for string and text data types. It can capture complex relationships between two field values by automatically customizing transformations for a given criterion. Points of note about this algorithm:
Does not return a 100% matching score on synonyms.
Strings that are similar, but contain characters like spaces or dashes do not return a 0% matching score.
When strings fall within the acceptable distance score they are not considered as the exact same. For example, a distance of 1 returns .95, a distance of 2 returns .9, etc.
A script task was added to run a matching operation from a workflow.
Fields can now have a weight of 0. Setting the weight to 0 excludes the fields from the pre-processing phase.
An API was added to check whether the add-on has any ongoing background operations.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
The options to specify what happens after a merged record is modified were removed. Now, the following behavior is applied when a record is in the merged state and the modified field:
Is not configured to use in matching operations: The add-on executes an automatic merge using the settings defined in the default matching policy's merge policy.
Is configured to use in matching operations: The record is changed to the <unset> state.
This release contains no changes to third-party libraries.
This release contains the following closed issues:
[MAME-7400] Changing record state to Unset fails when many transactions are committed in a short period of time.
[MAME-7455] The Evaluate matching and Run match services return inconsistent results.
[MAME-7479] There is conflict between the Evaluate matching and See details results.
[MAME-7542] Results cannot display after running Evaluate matching on a display with 1920X1080 resolution at 150% zoom.
This release contains the following known issues:
[MAME-7434] Linked records are duplicated in the link table when one of the foreign keys is not part of the primary key.
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: June 2022
This release contains no new features.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes in functionality.
This release contains no changes to third-party libraries.
[MAME-7532] A performance issue occurs during the grouping phase when running matching on an entire table.
This release contains the following known issues:
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: May 2022
This release contains no new features.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes in functionality.
The Spring framework was updated to version 5.2.22.
This release contains no closed issues.
This release contains the following known issues:
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: April 2022
This release contains no new features.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes in functionality.
This release contains no changes to third-party libraries.
This release contains the following closed issues:
[MAME-7397] The Merge view and the Lineage view cannot be accessed when all fields in a group or table are computed fields.
[MAME-7398] Performance is slow when fetching values n to n relationships.
[MAME-7436] When running the Evaluate matching service the field values do not display correctly when using the same field identities, but in different tables.
[MAME-7407] Boolean fields cannot be merged when Check null input is enabled.
This release contains the following known issues:
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: April 2022
This release contains no new features.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes in functionality.
The Spring framework was updated to version 5.2.20.
This release contains no closed issues.
This release contains the following known issues:
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: March 2022
Performance was improved to retrieve groups containing suspect records.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no changes in functionality.
This release contains the following changes to third-party libraries:
The Spring framework was updated to version 5.2.19.
The FasterXML/jackson-databind library was updated to version 2.13.2.1.
This release contains no closed issues.
This release contains the following known issues:
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: February 2022
This release includes the following new features:
You can now replicate matching metadata. This allows you to use third-party tools to access and query the metadata.
The following algorithms are now available:
Fuzzy full text: This algorithm works best for strings and can find the similar and fuzzy matches of the words in the compared values. It is based on the Levenshtein distance algorithm. The comparator takes into account the synonyms and stop words defined in the data model.
Range: This algorithm matches values within a predefined range. The two values are considered a match if the distance between two values is within the specified range.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains the following changes in functionality:
When Auto create new golden is enabled, the requirement was removed for the matching table's primary key to be an auto-incremented integer.
When executing add-on operations, only the add-on's table triggers are disabled. All user custom triggers remain enabled. Note that table triggers related to metadata linked fields are not recommended as it might affect the accuracy of match and merge operations.
This release contains no changes to third party libraries.
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
This release contains the following known issues:
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
For foreign key alignment, the FKs that are multi-value fields cannot be aligned (either manually or automatically).
Released: December 2021
This release includes no new features.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
The Apache Log4j library was upgraded to version 2.17.1.
This release contains no changes to third party libraries.
This release contains no closed issues.
This release contains the following known issues:
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
Released: November 2021
This release includes the following new features:
You can now exclude records from matching operations by specifying a field value. If a record contains the specified value, it is not included in matching operations. Using this functionality can help improve performance and improve the quality of matching results.
You can now modify records that were included in a merge operation.
The new Evaluate matching service allows you to test matching results. This can help you fine tune configuration settings to achieve desired results.
Adding an association to your data model allows you to view: a hierarchy of golden and merged records, and a tab when viewing golden record details that displays all merged records that target the golden.
Matching now supports use of non-Latin based text.
A new merge function on field survivorship allows you to add a constant value that is merged into golden records on survivorship.
The Full Text algorithm is now available to use in matching for fields with Text and String data types.
You can now use a foreign key field as a field in a decision tree's comparison node.
When upgrading from version 3.0.0, you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains the following functionality changes:
The default mode for Align foreign keys is now set to Manually.
The Case sensitivity option is now only available for fields with String and Text data types.
This release contains no changes to third party libraries.
This release contains no closed issues.
This release contains the following known issues:
When matching using the foreign key option, you can only specify a single hop foreign key. In other words, it cannot be a foreign key to a foreign key.
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
Released: September 2021
This release includes updates and enhancements to improve the performance of matching operations.
When upgrading from version 3.0.0, 3.1.0, or 3.1.1 you must update existing matching configurations. After updating configuration settings, you should re-execute matching to ensure that matching related metadata is up to date.
This release contains no functionality changes.
This release contains no changes to third party libraries.
This release contains no closed issues.
This release contains the following known issues:
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
Released: July 2021
This release includes the following new features:
A new REST API is available to perform matching related operations. You can use Swagger to view the API documentation. See REST overview for more information.
Matching operations after record submission are now available for records created using the EBX® API. There are some limitations to this functionality. See the Known issues section below for details.
Several matching services are now available to use in workflows and perspectives. See Perspectives and workflows for more information.
The add-on now supports link tables when matching with relationships. Additionally, to avoid confusion with EBX®'s concpet of associations, this feature was renamed to Use relationships.
The add-on's matching algorithm library now includes the Soundex algorithm. This algorithm works well for short strings such as names. See Matching algorithms for more information.
The list of algorithms is now filtered based on the search strategy configured for a matching field. Additionally, the validation report will display a warning if the configured algorithm is not compatible with the field's search strategy. See Matching algorithms for more information.
Administrators can now set a matching policy as the default policy from the Matching policies tab in the Actions menu.
A merge policy's record selection mode now includes the First acquired option to select the oldest record during survivorship.
The Customize source value for new golden now supports foreign keys.
Matching metadata can now be added to tables by creating a custom view. See Displaying matching metadata for more details.
The add-on's Consolidated view the Actions menu where you can execute matching related operations.
When viewing table data, you can now access the Metadata view's Lineage tab by selecting View lineage.
When a group contains only one record that does not need to be merged, a new Set as golden button allows you to set its state to golden.
This release contains the following functionality changes: When using the Run match service to run a match on all records in a table, you must now select all records in the table instead of selecting nothing.
This release contains no changes to third party libraries.
This release contains no closed issues.
This release contains the following known issues:
The Validation service in the Consolidated view runs as expected when first executed, but slow on subsequent executions.
In order to accurately track record lineage, table history must remain activated over time. If the table's activation status repeatedly changes, the lineage data will be inconsistent.
Severe errors occur when multiple datasets based on the same model exist and a table from one of these datasets is activated in an add-on matching configuration. As a workaround, you can create another data model publication using the Manage publications service in the Data Model Assistant. After creating a dataset for the new publication, you can follow the normal procedures to configure a matching policy for tables in the new dataset.
When matching is set to After submission and batch creation of records occurs using the EBX® API, the number of submitted or created records is limited to 2000 records in an API call for tables that have a single primary key. For a composite primary the limit is divided by the number of fields that makes up the primary key. For example, with a table that has a primary key comprised of 2 fields, the limit would be 1000 records. When your requirements exceed these limits, it is recommended that you pause the matching trigger, load the data, and resume matching. Also, note that when matching is triggered the dataspace is locked until matching completes. Any incoming requests during this time are blocked and will return errors.
Released: March 2021
[MAME-5857] A documentation issue was corrected.
Released: March 2021
The TIBCO EBX® Match and Merge Add-on locates and merges duplicate data values. The goal is to obtain a singular record that most accurately defines a business entity. The add-on identifies these as golden records.
Features available to administrators in include the ability to:
Make data available to the add-on by registering and activating matching on tables.
Configure a matching policy that determines how the add-on compares data. A matching policy also specifies whether users must start the matching process manually or if it starts automatically when data is created or updated.
Create a merge policy that establishes how duplicate data values are merged into a golden record. Merge policies can also execute automatically after a matching process runs.
Set up trusted sources that define a hierarchy of data sources. The add-on uses the source's trust level to during matching to choose the most accurate record and during a merge to select values to merge.
Administrators can use Administrator Guide to learn more about configuration options available in this release.
Features available to business users include the ability to:
Select data and initiate matching and merge operations. Search for records that might match when creating a new record.
Manage records that grouped together after running a matching process. The add-on groups records that are suspected of containing duplicate data or are positively identified as being duplicate. When managing groups you can also specify that a record is a false-positive match so that it no longer matches with the records in its group.
View record metadata applied by the add-on to categorize records. Two different views allow users to view individual record, or an entire table's metadata. Some information displayed includes the record's history, lineage, state, and records identified as false positive matches.
Business users can learn more about available actions and view in the User Guide.