Parameters

conf_qlets

this is an array of the input "Confidence" querylets. The ConfidenceQlt class adds the three additional parameters to each query: minimum confidence measure, match score cutoff and the no prefilter flag.

invalid_only

If true the logic for this combiner in inverted: This first querylet whose confidence is at or below the confidence threshold is used.

Return Value

a NetricsQuery object for the First Valid combiner.

This creates a First Valid score combiner. This combiner is intended for use with Rlink queries. The Rlink query returns a confidence measure that is based on how well trained the model is for the particular case presented. A low confidence measure indicates a low confidence in the reliability of the score output. When the confidence is low it may be desirable to employ a different query for this particular comparison. The First Valid score combiner provides a means of doing so.

Note that all current score generators produce a confidence measure of 1.0 (the highest possible). All score combiners, except Rlink and the First Valid score combiner, return the minimum confidence score of their input querylets as their confidence score. The Rlink combiner produces a confidence score based on how well trained the Learn Model is for the example evaluated. The First Valid score combiner returns the confidence measure of the selected querylet.

The First Valid score combiner allows you to assign a minimum confidence measure to each querylet. The querylets are scanned in order, the first querylet that meets or exceeds its minimum confidence measure is used as the output of the First Valid combiner. Note that it is the first querylet that meets the criteria that is selected, not the querylet with the highest confidence measure. If no querylet meets its minimum confidence measure the First Valid combiner returns a reject score (-1.0), so the record is not returned.

The standard use case is to provide an alternative query to be used for those cases where the Learn Model is poorly trained and may not be reliable. The first querylet is the Rlink query employing the Learn Model, the second querylet is a fixed query that does not use a Learn Model. When the Learn Model is unsure of its results, as indicated by a confidence measure below a given cutoff value, the First Valid combiner would return the results of the alternative query. When the Learn Model is sure of its prediction, as indicated by a confidence measure at or above the given cutoff, the results of the Learn Model are used. In this way an incompletely trained Learn Model can be safely used. You get the benefit of the greater accuracy of the Learn Model for those situations it has been trained for, you get the results of a predefined query for those situations where the Learn Model has not been well trained. In theory a number of alternative queries could be employed that use different Learn Models.

The meaning of a particular score may be different depending on which querylet returned the score. For example a score of 0.6 from a Learn Model typically indicates a match between the two items. The same 0.6 score from a standard AND of simple and cognate queries generally indicates a fairly poor match. To correct for this difference in the meaning of the scores the First Valid combiner "normalizes" the scores to a common score range. For each querylet a match score cutoff must be provided. This score represents the level that forms the boundary between scores that represent a match and those that do not. If an alternative querylet is selected the scores for that querylet are normalized such that its match score cutoff is set equivalent to the match score cutoff of the first querylet. So in a typical case the Learn Model querylet may have a cutoff of 0.5, and the fixed querylet a cutoff of 0.85. If the fixed query is selected and its match score is 0.85, the score returned by the First Valid combiner is adjusted to 0.5. A score of 0.8 is adjusted to about 0.47 and a score of 0.99 is adjusted to about 0.96. This adjustment of scores allows a common cutoff score, or other processing rules based on scores, to be applied regardless of which querylet is selected by the First Valid combiner.

As the input querylets to the First Valid score combiner likely use the same match data, it is often unnecessary and wasteful to apply all of the querylets in the prefilter stage of matching. A no prefilter flag is provided for each querylet. If set the querylet is ignored by the prefilter. This can dramatically improve performance and may help in accuracy. Note it is invalid to set the no prefilter flag for all querylets. At least one must be passed to the prefilter.

As a combination of a query and a record that returns a low confidence from a Learn Model forms an ideal training pair for the model. Training on this pair can fill in the poorly train region, allowing the Learn Model to predict with confidence the next time such an example is encountered. To facilitate retrieving such pairs the invalid only flag can be set to true. This reverses the sense of the First Valid combiner so that it returns only records that failed to meet the confidence criteria for at least one querylet. All other records are assigned the reject score of -1.0.