Package com.orchestranetworks.addon.daqa.cleansing

Provides classes and interfaces required for cleansing operation.

See: Description

Interface Summary

Interface	Description
CleansingController	Provides the entry point for the execution of all cleansing procedures and operations.
CleansingExecutionResult	Provides results from a cleansing procedure or cleansing operation execution.
CleansingOperationDefinition	Defines a cleansing operation.
CleansingOperationDefinitionContext	Represents a cleansing operation's configuration.
CleansingProcedureDefinitionContext	Represents a cleansing procedure configuration.
CleansingState	Represents a Cleansing state.
CustomSelectionType	Provides the list of custom selection types.
HTMLComponentType	Provides the list of supported HTML component types.

Class Summary

Class	Description
BigDataValues	Defines a set of values to be stored in the Big data report tables.
CleansingControllerFactory	Defines the factory of CleansingController.
CleansingExecutionContext	Provides an execution context for the cleansing procedure or cleansing operation.
CleansingExecutionEnvironment	Defines an execution environment for a cleansing procedure or cleansing operation.
CleansingOperationView	Provides a user interface is required when fixing quality defects found during a cleansing operation.
CleansingProcedureDefinition	Defines a cleansing procedure.
CleansingProcedureDefinitionCatalog	Registers cleansing procedures.
CleansingRecord	Supports cleansing metadata storage.
CleansingReport	Java Bean container for the outcome of a CleansingProcedure or CleansingOperation.
CleansingStatisticsProcedure	Calculates statistics related to a cleansing procedure.
DECType	Represents the Data Element Concept type.
DeprecatedRecordChecker	Extends this class to define a Java class to check deprecated records.
InputParameter	Represents the input parameters for a cleansing procedure or cleansing operation.
ListConstraint	Defines constraints for multi-occurs parameters.
ParameterDefinition	Represents the definition of parameter for a cleansing procedure or cleansing operation.

Enum Summary

Enum	Description
CleansingExecutionContext.ExecutionMode	Defines cleansing procedure execution mode.

Package com.orchestranetworks.addon.daqa.cleansing Description

Provides classes and interfaces required for cleansing operation.

Cleansing operation

To create a cleansing operation, its definition and view class must be defined.

Definition class:

        public final class TestCleansingOperationDefinition implements CleansingOperationDefinition
        {
                public UserMessage getName()
                {
                        return UserMessage.createInfo("Test Cleansing operation");
                }
        
                public UserMessage getDescription()
                {
                        return null;
                }
        
                public List <ParameterDefinition> getInputParameters()
                {
                        return null;
                }
        
                public boolean isMatchingActive()
                {
                        return false;
                }
                public UserMessage validateCleansingOperationInputParameters(
                List <InputParameter> inputParameters)
                {
                        return null;
                }
                public CleansingOperationView getOperationView()
                {
                        return new TestCleansingOperationView();
                }
        }

View class:

        public final class TestCleansingOperationView extends CleansingOperationView
        {
                public void displayView() throws DaqaException
                {
                        ServiceContext serviceContext = this.getServiceContext();
                        UIServiceComponentWriter writer = serviceContext.getUIComponentWriter();
                        writer.add("Draw all UI items of the Cleansing operation view here.");
                }
        
                public CleansingReport executeOperation(CleansingExecutionEnvironment environment)
                        throws DaqaException
                {
                        //Provide the execution of the Cleansing operation.
                        BigDataValues bigDataValues = new BigDataValues();
                        bigDataValues.add(
                                CleansingReportPaths._BigDataCleansingReport._CleansingProcedureFields_NumberOfRecordsToFix,
                                Integer.valueOf(1000));
                        bigDataValues.add(
                                CleansingReportPaths._BigDataCleansingReport._CleansingProcedureFields_NumberOfFixedRecords,
                                Integer.valueOf(100));
        
                        CleansingReport cleansingReport = new CleansingReport();
                        cleansingReport.setExecutionDate(new GregorianCalendar().getTime());
                        cleansingReport.setBigDataValues(bigDataValues);
                        return cleansingReport;
                }
        }

Cleansing procedure

To create a cleansing procedure, its definition and implementation class must be defined.

Definition class:

        public final class TestCleansingProcedureDefinition extends CleansingProcedureDefinition
        {
                public String getCode()
                {
                        return "Test-CP";
                }
        
                public UserMessage getName()
                {
                        return UserMessage.createInfo("Test Cleansing procedure");
                }
        
                public UserMessage getDescription()
                {
                        return null;
                }
        
                public DECType getDECType()
                {
                        return DECType.FIELD;
                }
        
                public boolean canBeUsedInProfilingProcedure()
                {
                        return false;
                }
        
                public List<ParameterDefinition> getInputParameters()
                {
                        List<ParameterDefinition> inputs = new ArrayList<ParameterDefinition>();
                        List<String> defaultEmptyValues = new ArrayList<String>();
                        defaultEmptyValues.add("");
                        defaultEmptyValues.add("null");
                         inputs.add(new ParameterDefinition(
                "valueAsEmpty",
                SchemaTypeName.XS_STRING,
                UserMessage.createInfo("Values considered as empty"),
                UserMessage.createInfo("Values considered as empty"),
                new ListConstraint(1, Integer.MAX_VALUE, defaultEmptyValues)));
                        return Collections.unmodifiableList(inputs);
                }
        
                public boolean canBeUsedForManyTables()
                {
                        return false;
                }
        
                public boolean canBeUsedForManyFields()
                {
                        return false;
                }
        
                public List<CleansingOperationDefinition> getCleansingOperations()
                {
                        //Register all operations for the Cleansing procedure here.
                        //One Cleansing procedures can have 0 or many Cleansing operations.
                        List<CleansingOperationDefinition> operations = new ArrayList<CleansingOperationDefinition>();
                        operations.add(new TestCleansingOperationDefinition());
                        return operations;
                }
        
                public CleansingStatisticsProcedure getCleansingProcedureImplementation()
                {
                        return new TestCleansingProcedureImplementation();
                }
        }

Implementation class:

        public final class TestCleansingProcedureImplementation extends CleansingStatisticsProcedure
        {
                public List<CleansingReport> calculate(CleansingExecutionContext context)
                {
                        if (context.getExecutionMode().isProcedureExecutionMode())
                        {
                                List<CleansingReport> cleansingReports = new ArrayList<CleansingReport>();
                                CleansingReport cleansingReport = new CleansingReport();
                                BigDataValues bigDataValues = new BigDataValues();
                                bigDataValues.add(
                                        CleansingReportPaths._BigDataCleansingReport._CleansingReportGenericFields_Dataspace,
                                        context.getExecutionEnvironment().getHome().getKey().format());
                                bigDataValues.add(
                                        CleansingReportPaths._BigDataCleansingReport._CleansingReportGenericFields_Dataset,
                                        context.getExecutionEnvironment().getDataset().getAdaptationName().getStringName());
                                bigDataValues.add(
                                        CleansingReportPaths._BigDataCleansingReport._CleansingReportGenericFields_TablePath,
                                        context.getExecutionEnvironment().getTable().getTablePath().format());
                                bigDataValues.add(
                                        CleansingReportPaths._BigDataCleansingReport._CleansingProcedureFields_NumberOfRecordsToFix,
                                        Integer.valueOf(1000));
                                cleansingReport.setBigDataValues(bigDataValues);
                                cleansingReports.add(cleansingReport);
                                return cleansingReports;
                        }
                        else
                        //case: Profiling mode
                        {
                                List<CleansingReport> cleansingReports = new ArrayList<CleansingReport>();
                                CleansingReport cleansingReport = new CleansingReport();
                                BigDataValues bigDataValues = new BigDataValues();
                                bigDataValues.add(
                                        CleansingReportPaths._BigDataProfilingReport._CleansingReportGenericFields_Dataspace,
                                        context.getExecutionEnvironment().getHome().getKey().format());
                                bigDataValues.add(
                                        CleansingReportPaths._BigDataProfilingReport._CleansingReportGenericFields_Dataset,
                                        context.getExecutionEnvironment().getDataset().getAdaptationName().getStringName());
                                bigDataValues.add(
                                        CleansingReportPaths._BigDataProfilingReport._CleansingReportGenericFields_TablePath,
                                        context.getExecutionEnvironment().getTable().getTablePath().format());
                                bigDataValues.add(
                                        CleansingReportPaths._BigDataProfilingReport._ProfilingFields_NumberOfRecordsToFix,
                                        Integer.valueOf(1000));
                                cleansingReport.setBigDataValues(bigDataValues);
                                cleansingReports.add(cleansingReport);
                                return cleansingReports;
                        }
                }
        }

Finally, the cleansing procedure must be registered with the add-on in order to be available in the configuration:

        CleansingProcedureDefinitionCatalog.add(new TestCleansingProcedureDefinition());