Managing Rules

ibi Data Quality provides users with a set of built-in rules that can be used for Data Quality (DQ) analysis. These rules provide sample implementations illustrating the use of service parameters for the different services in ibi Data Quality. Users can add, edit, or delete rules in the Rules Catalog.

Adding New Rules

There are two ways to add new rules.

Defining New Rule

To define a new rule:

Go to the Rules tab (Rules Catalog) and click on New Rule, as shown in the following image.
Enter the rule information.
1. A unique name for the rule.
2. Set Active status to True if you want to publish the rule for active use.
  Note: If you accept the default status as False, then the rule cannot be used for DQ analysis.
3. A brief description of the rule.
4. Add rule metadata (optional):
  - Content Group. Select from the list of available values or add a new value. Content groups are a higher level classification of data classes. For example, data classes email and phone_number belong to content group contact, us_ssn and us_dea belong to content group identifier.
  - Data Class. Select a data class from the list of available data classes.
  - Entity. Select one or more values from the list of available entities or add a new value.
  - Country. Select one or more countries from the list of available countries.
  - Industry. Select one or more industries from the list of available industries.
Enter service information.
- Select workspace.DQ Core represents prebuilt services delivered with the product. Custom represents services registered by your developers in this ibi Data Quality instance.
- Select service. Search and select the appropriate service. Depending on the service you select, you will be presented with service parameters to set.
- Service parameters. These are service-specific parameters. Refer to service parameter definitions on how to configure them.
Enter ticketing criteria to define scenarios where a ticket can be generated for data quality issues that require manual intervention. If multiple criteria are met, only one ticket is created that includes a description of each issue identified during DQ analysis.
Note: Tickets generated during runtime execution of a Rule are stored in the analysis results folder in a file called tickets.json. The calling application is responsible for processing the ticket through a ticket management workflow.
1. Tag based criteria - You can select one or more tag values and specify a minimum threshold (number or percentage of records). When records in a data meet this minimum threshold for this tag value, a data quality ticket will be generated with the description of the issue identified through this criteria.
2. Tag Category based criteria - You can select one or more tag categories and specify a minimum threshold (number or percentage of records). When the records in a data set meet this minimum threshold for this tag category, a data quality ticket will be generated with the description of the issue identified through this criteria.
3. Score based criteria - You can specify a target DQ score between 1 and 100. When records in a data set do not meet this minimum DQ score, a data quality ticket will be generated with the description of the issue identified through this criteria.
4. Optional User Comment - You can provide an optional comment for the data governance application or the end user who is going to resolve this data quality issue.
Click Finish and verify the rule is available for use in the catalog.

Verifying a New Rule

To verify a new rule:

Go to the Rules tab (Rules Catalog) and verify that the rule is available for use in the catalog. Ensure the active flag is set to True, as shown in the following image.
Upload data and attach the rule to a data variable to execute the rules-based DQ analysis, as shown in the following image.
Review Rule analysis Results and Score, as shown in the following two images.
Review tickets generated from the Rule, as shown in the following image.

Did you find this helpful?

Yes No