com.netrics.likeit

Class NetricsQuery

java.lang.Object

com.netrics.likeit.NetricsQuery

All Implemented Interfaces:

java.io.Serializable

public class NetricsQuery
extends java.lang.Object
implements java.io.Serializable

NetricsQuery is used to implement the complex query structures introduced with release 4.1. Queries take the form of a hierarchial tree. There are currently four ways to create basic scores - a simple query comparison, a cognate query comparison, a date comparison, and a predicate querylet. This will be explained in further detail below. These scores can then be combined by five more operators - AND, OR, NOT, WGTBYFIELD and RLINK (this last is how to use the Decision Engine model provided to you by netrics if you purchased the Decision Engine). For each of the score combiners, you will need to pass the NetricsQuery "children" which will be combined by that operator. It follows that all the "leaves" in the tree will need to be score generators, and all the "branches" will need to be score combiners.

To create the most basic search, just follow the following code:

        NetricsSearchCfg tblCfg = new NetricsSearchCfg("test");
        tblCfg.setNetricsQuery(NetricsQuery.Simple("Michael Phelps",null,null));

        NetricsSearchResponse resp = si.search(tblCfg, null);

The above code will search the table "test" (all fields) for the string "Michael Phelps." It's just that easy. A much more complicated search might look like the following:

        String []fnames = {"name1"};
        NetricsQuery nq1 = NetricsQuery.Simple("rec1f1",fnames,null);

        String []queries = {"rec1f2","rec1f1"};
        String []fnames2 = {"name1","name2"};
        NetricsQuery nq2 = NetricsQuery.Cognate(queries,fnames2,null,.8);

        String []fnames3 = {"date"};
        NetricsQuery nq3 = NetricsQuery.Custom("10/12/2001",fnames3,null,NetricsQuery.CS_DATE);

        NetricsQuery nq4 = NetricsQuery.Predicate("DATE \"2001/01/01\" <= $\"date\" and $\"date\" <= DATE \"2001/12/31\"") ;

        NetricsQuery []nqs = {nq1,nq2,nq3,nq4};
        NetricsQuery nq = NetricsQuery.And(null,nqs);

The above query combines 4 different querylets using an AND (which performs a simple average of the individual scores). The first querylet is a simple string comparison, the second is a cognate string comparison, the third is a date comparison, and the fourth is a predicate comparison. Look below to find out more about each kind of query.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static int	CS_DATE Custom Date comparison scorer
static int	CS_NONE Not a custom scorer

Constructor Summary

Constructors

Constructor and Description
NetricsQuery()

Method Summary

Methods

Modifier and Type	Method and Description
static NetricsQuery	And(double[] weights, NetricsQuery[] nqs) Create an AND Query Expression Node This is used to combine multiple NetricsQuery scores into a single score.
static NetricsQuery	And(double[] weights, NetricsQuery[] nqs, double[] ignore_scores) Create an AND Query Expression Node with ignore scores.
static NetricsQuery	And(double[] weights, NetricsQuery[] nqs, double[] ignore_scores, double[] reject_scores) Create an AND Query Expression Node with ignore scores and reject scores.
static NetricsQuery	Attributes(java.lang.String[] attr_values, java.lang.String[] attr_names, double[] attr_weights) Create a Variable Attributes Query node.
static NetricsQuery	Cognate(java.lang.String[] qstrs, java.lang.String[] fldnames, double[] fldweights, double noncogwgt) Create a Cognate Query Expression Node.
static NetricsQuery	Custom(java.lang.String qstr, java.lang.String[] fldnames, double[] fldweights, int customtype) Create a Custom Query node.
void	NetricsQuery() Create an empty Query Node.
static NetricsQuery	Not(NetricsQuery nq) Create a NOT Query Expression Node.
static NetricsQuery	Or(double[] weights, NetricsQuery[] nqs) Create an OR Query Expression Node.
static NetricsQuery	Predicate(NetricsPredicate pred) Create a Predicate Query Expression Node.
static NetricsQuery	Predicate(java.lang.String expr) Create a Predicate Query Expression Node from a string predicate.
static NetricsQuery	Rlink(java.lang.String modelname, NetricsQuery[] nqs) Create an RLINK Query Expression Node.
void	scoreType(int scoreType) This specifies the type of score to be used for ordering records.
void	setEmptyScore(double score) Sets the score a comparison gets when empty data is encountered.
void	setInvalidScore(double score) Sets the score a comparison gets when invalid data is encountered.
void	setXparm(int id, double value) Use this method only on the advice of your Netrics representative.
void	setXparm(int id, int value) Use this method only on the advice of your Netrics representative.
static NetricsQuery	Simple(java.lang.String qstr, java.lang.String[] fldnames, double[] fldweights) Create a Simple Query Expression Node.
void	useEphemeralThesaurus(com.netrics.likeit.NetricsBaseThesaurus thes_def, double theswgt) Assign an ephemeral thesaurus to be used by this NetricsQuery.
void	useThesaurus(java.lang.String thesname, double theswgt) Assign a thesaurus to be used by the NetricsQuery object.
static NetricsQuery	Wgtbyfield(java.lang.String fieldName, NetricsQuery nq) Create a weighted field query expression node.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CS_NONE

public static int CS_NONE

Not a custom scorer

CS_DATE

public static int CS_DATE

Custom Date comparison scorer

Constructor Detail

NetricsQuery

public NetricsQuery()

Method Detail

NetricsQuery

public void NetricsQuery()

Create an empty Query Node.

Custom

public static NetricsQuery Custom(java.lang.String qstr,
                  java.lang.String[] fldnames,
                  double[] fldweights,
                  int customtype)

Create a Custom Query node.

Currently there is only one custom query type and that is NetricsQuery.CS_DATE. To perform a date search, pass the query date in "qstr," the single date field in the "fldnames" vector, set fldweights to null, and set the customtype to NetricsQuery.CS_DATE. A date comparison will look for inversions between month and date, misplaced century information, and other qualities that a normal string comparison would not take into account.

Parameters:

qstr - the query string

fldnames - list of fields to query

fldweights - list of weights for field

customtype - type of the custom query

Simple

public static NetricsQuery Simple(java.lang.String qstr,
                  java.lang.String[] fldnames,
                  double[] fldweights)

Create a Simple Query Expression Node.

This is the basic Netrics query and is used to compare an arbitrary query string against a field set in a table.

Parameters:

qstr - the query string

fldnames - list of fields to query

fldweights - list of weights for field

And

public static NetricsQuery And(double[] weights,
               NetricsQuery[] nqs)

Create an AND Query Expression Node

This is used to combine multiple NetricsQuery scores into a single score. This is usually performed with a simple average, but can be a weighted average if the user utilizes the weights parameter.

Parameters:

weights - list of floats that are the weights for the sub expressions of this query

nqs - array of NetricsQuery objects that are the sub expressions for this AND

And

public static NetricsQuery And(double[] weights,
               NetricsQuery[] nqs,
               double[] ignore_scores)

Create an AND Query Expression Node with ignore scores.

This is used to combine multiple NetricsQuery scores into a single score. This is usually performed with a simple average, but can be a weighted average if the user utilizes the weights parameter.

This version allows the caller to specify a set of scores, one per sub expression, that define a lower bound for the sub expression score. If the score is below the specified lower bound the sub expression will be ignored completely when computing the average or weighted average for the output score.

There are two special score values: -1.0 is the special score used to indicate a record should be rejected. Typically this is used to reject records when a search on empty or invalid data is performed (see the methods setInvalidScore and setEmptyScore for how to change the score assigned in such cases). Instead of rejecting a record a sub expression can be ignored if empty or invalid data is encountered by setting the empty and invalid scores to -1.0 and then using the ignore threshold to ignore scores of -1.0. -2.0 can be used as an indicator that no ignore threshold processing is to be performed for this sub expression.

A cautionary note on ignore scores: Ignore threshold scores should be used with extreme caution. Ignoring a low score in one sub expression will likely boost the score of a record with a poor match on that sub expression (i.e. a score below the threshold) above the score of a similar record with a good match on that sub expression (i.e. above the threshold). Thus when used inappropriately ignore score thresholds will tend to push poorer matches above better matches. This is especially true if ignore thresholds are applied to more than one sub expression. Thus ignore thresholds should only be applied in the rare cases where a low score indicates the sub expression is completely irrelevant and the record should score above close but imperfect matches on the sub expression. Another valid use would be to ignore empty fields as described above, although normally it is better to just let the default score of 0.0 be averaged into the output score.

Parameters:

weights - list of floats that are the weights for the sub expressions of this query

nqs - array of NetricsQuery objects that are the sub expressions for this AND

ignore_scores - array of floats that are the ignore-scores for the sub expressions for this AND

See Also:

setEmptyScore(double), setInvalidScore(double)

And

public static NetricsQuery And(double[] weights,
               NetricsQuery[] nqs,
               double[] ignore_scores,
               double[] reject_scores)

Create an AND Query Expression Node with ignore scores and reject scores.

This is used to combine multiple NetricsQuery scores into a single score. This is usually performed with a simple average, but can be a weighted average if the user utilizes the weights parameter.

This version allows the caller to specify both a set of ignore scores and a set of reject scores. Ignore scores define a lower bound for the sub expression score. If the score is below the specified lower bound the sub expression will be ignored completely when computing the average or weighted average for the output score. Reject scores define a lower bound also. But in the case of the reject score if the sub expression score is below the reject score the entire record will be rejected. (More precisely the record is assigned a score of -1.0, which causes the record to be rejected unless the score is ignored in a higher level And expression.)

Both ignore scores and reject scores support two special score values: -1.0 is the special score used to indicate a record should be rejected. Typically this is used to reject records when a search on empty or invalid data is performed (see the methods setInvalidScore and setEmptyScore for how to change the score assigned in such cases). Instead of rejecting a record a sub expression can be ignored if empty or invalid data is encountered by setting the empty and invalid scores to -1.0 and then using the ignore threshold to ignore scores of -1.0. -2.0 can be used as an indicator that no ignore or reject threshold processing is to be performed for this sub expression.

If either of the ignore scores or reject scores are null then no ignore or reject tests respectively are performed for this And. If a sub expression is assigned both an ignore score and a reject score then the lesser value takes precedence over its range, the greater value is applied to the range from the lesser to the greater. E.g. if the reject score for sub expression 1 is 0.2 and the ignore score for sub expression 1 is 0.4 then records with a sub expression 1 score less than 0.2 are rejected and records with a score from 0.2 to 0.4 will ignore sub expression 1. Conversely if the ignore score is 0.2 and the reject score 0.4 then scores less than 0.2 will cause sub expression 1 to be ignored whereas scores from 0.2 to 0.4 will cause the record to be rejected.

A cautionary note on ignore scores: Ignore threshold scores should be used with extreme caution. Ignoring a low score in one sub expression will likely boost the score of a record with a poor match on that sub expression (i.e. a score below the threshold) above the score of a similar record with a good match on that sub expression (i.e. above the threshold). Thus when used inappropriately ignore score thresholds will tend to push poorer matches above better matches. This is especially true if ignore thresholds are applied to more than one sub expression.

Further care is needed when using both ignore thresholds and reject thresholds on the same set of sub expressions. Ignore thresholds tend to favor records with high scores for one expression and very low scores for the others. Reject thresholds will pass a record only if all sub expressions are above the given threshold values. Thus combining ignore thresholds and reject thresholds can result in few or no records being returned.

Parameters:

weights - list of floats that are the weights for the sub expressions of this query

nqs - array of NetricsQuery objects that are the sub expressions for this AND

ignore_scores - array of floats that are the ignore-scores for the sub expressions for this AND

reject_scores - array of floats that are the reject-scores for the sub expressions for this AND

See Also:

setEmptyScore(double), setInvalidScore(double)

Or

public static NetricsQuery Or(double[] weights,
              NetricsQuery[] nqs)

Create an OR Query Expression Node.

This is used to choose between multiple NetricsQuery scores by taking the maximum score. To weight the individual scores, the user can utilize the weights parameter.

Parameters:

weights - list of floats that are the weights for the sub expressions of this query

nqs - list of NetricsQuery objects that are the sub expressions for this OR.

Not

public static NetricsQuery Not(NetricsQuery nq)

Create a NOT Query Expression Node.

Take the compliment of the NetricsQuery score - e.g. a .1 becomes a .9.

Parameters:

nq - NetricsQuery object that is the sub expression for this NOT.

Wgtbyfield

public static NetricsQuery Wgtbyfield(java.lang.String fieldName,
                      NetricsQuery nq)

Create a weighted field query expression node.

This replaces record weights from earlier versions of Netrics software. This is used to modify the score of a match based on a value in the record that was matched. It provides a way of favoring certain records over others. Essentially the score of the passed in NetricsQuery is multiplied by the value in the indicated field of the record. The indicated field must have a field type of FLOAT. If the resulting score is greater than 1.0 it is set to 1.0, if it is less than 0.0 it is set to 0.0.

Attributes

public static NetricsQuery Attributes(java.lang.String[] attr_values,
                      java.lang.String[] attr_names,
                      double[] attr_weights)

Create a Variable Attributes Query node.

This is used to query a set of Variable Attributes values. It is equivalent to an And of a set of simple queries on the attribute values with an empty score of -1.0 and ignore scores of -1.0 on the And. That is it is an And of a match on each of the listed attributes where missing attribute values in the record are ignored.

The same options excepted by a Simple query can be applied to this query. That includes using a thesaurus, which will apply to all attributes, and setting the empty score. Note that by setting the empty score to something other than -1.0 the default behavior of ignoring missing attribute values can be changed so that missing values will lower the overall score rather than being ignored.

The attr_weights argument allows a weight factor to be applied to each of the attributes. This behaves as the querylet weights do in the And query, it adjusts the relative importance of a particular attribute value but it does not reduce the final score.

Parameters:

attr_values - the list of attribute values to be matched.

attr_names - the list of names of the attribute to be matched. This must correspond to the attr_values.

attr_weights - the list of weighting factors for each attribute. This may be null, in which case all attributes get equal weights. If given this must match the attr_values and attr_names arrays.

Returns:

A new NetricsQuery object for this query.

Rlink

public static NetricsQuery Rlink(java.lang.String modelname,
                 NetricsQuery[] nqs)

Create an RLINK Query Expression Node.

This is only available if you purchased the Netrics Decision Engine. RLINK is the intelligent score combiner that can be used in lieu of an AND, and is a machine learning model that should be set up exactly as it was trained. Follow the instructions of your Netrics contact in order to use this NetricsQuery type.

Parameters:

modelname - the name of the decision engine model to use

nqs - list of NetricsQuery objects that are the sub expressions for this RLINK expression.

Cognate

public static NetricsQuery Cognate(java.lang.String[] qstrs,
                   java.lang.String[] fldnames,
                   double[] fldweights,
                   double noncogwgt)

Create a Cognate Query Expression Node.

A cognate query is a more complicated version of a Simple query. It uses the same string comparison algorithm but performs the match with multiple query strings. For instance, a first and last name can be searched against the first and last name fields of the database. This aids in finding transposted fields (where a first name occurs in the last name field or vice versa). The length of the qstrs array must always equal the length of the fldnames array. The fldweights array must also be the same length if it is given. The noncogwgt is the weight to assign transposed field matches (if noncogwgt is set to .8, the score of a first name match in the last name field would be reduced by 80%).

Parameters:

qstrs - the list of query strings

fldnames - list of fields to query

fldweights - list of weights for field

noncogwgt - penalty applied to diagonal (non-cognate) field matches

useThesaurus

public void useThesaurus(java.lang.String thesname,
                double theswgt)
                  throws NetricsException

Assign a thesaurus to be used by the NetricsQuery object.

The thesaurus and the query fields should use the same character map.

Parameters:

thesname - the name of the thesaurus

theswgt - the weight to give thesaurus matches

Throws:

NetricsException

useEphemeralThesaurus

public void useEphemeralThesaurus(com.netrics.likeit.NetricsBaseThesaurus thes_def,
                         double theswgt)
                           throws NetricsException

Assign an ephemeral thesaurus to be used by this NetricsQuery.

An ephemeral thesaurus is one that exists only for the duration of a single query and is accessible only by that query. The thesaurus name, although required for consistency, is not used. Thus ephemeral thesauri may have the same name as a permanent thesaurus or another ephemeral thesaurus without causing interference.

Ephemeral thesauri are intended for those cases where possible substitutions or weighted terms for a query are generated dynamically based on the query and is not a fixed set of substitutions or weightings that can be encoded into a static thesaurus.

Parameters:

thes_def - A thesaurus object (an object of any class extending NetricsBaseThesaurus).

theswgt - the weight to give thesaurus matches

Throws:

NetricsException

See Also:

NetricsThesaurus, NetricsWeightedDictionary, NetricsCombinedThesaurus

setXparm

public void setXparm(int id,
            int value)
              throws NetricsException

Use this method only on the advice of your Netrics representative.

Throws:

NetricsException

setXparm

public void setXparm(int id,
            double value)
              throws NetricsException

Use this method only on the advice of your Netrics representative.

Throws:

NetricsException

Predicate

public static NetricsQuery Predicate(NetricsPredicate pred)

Create a Predicate Query Expression Node.

The predicate will be evaluated on the record and if it's true, the NetricsQuery will be given a score of 1.0. If it's false a 0.0 score is used. This is a simple comparison for use when other queries are both slower and more cumbersome (like a gender comparison). It can also be used when inexact matching is not required.

The value returned by the predicate expression must be either a boolean value or a floating point value in the range 0.0 to 1.0. If a floating point value is returned it is used as the score for this query expression.

Parameters:

pred - the NetricsPredicate object to evaluate

Predicate

public static NetricsQuery Predicate(java.lang.String expr)

Create a Predicate Query Expression Node from a string predicate.

Parameters:

expr - the string predicate expression

setInvalidScore

public void setInvalidScore(double score)

Sets the score a comparison gets when invalid data is encountered.

A comparison will receive this score when invalid data in the record is encountered or certain other errors occur when matching the record. The special score -1.0 can be used to indicate the record is to be rejected. The default setting is appropriate for most situations, thus it is rarely necessary to set this.

Parameters:

score - the score to be assigned.

setEmptyScore

public void setEmptyScore(double score)

Sets the score a comparison gets when empty data is encountered.

A comparison will receive this score when empty data is encountered. The special score -1.0 can be used to indicate the record should be rejected. The default setting is appropriate for most situations, thus it is rarely necessary to set this.

Parameters:

score - the score to be assigned.

scoreType

public void scoreType(int scoreType)

This specifies the type of score to be used for ordering records.

Score types include the following:

Normal (pass in NetricsSearchOpts.SCORE_NORMAL)
This is the standard Netrics search. This type of search looks for the query text inside the record text. The presence of extra information in the record not found in the query does not penalize the record score. Use this score type for a substring or keyword search.

Symmetric (pass in NetricsSearchOpts.SCORE_SYMMETRIC)
A symmetric search compares the full texts of both the query and record and evaluates their similarity. If the record contains information not present in the query, the score will be lower than if that information had not been present. Use this score only when the query represents the entirety of the text expected to be found.

Reverse (pass in NetricsSearchOpts.SCORE_REVERSE)
Reverse scoring functions the same as the normal search, but with the roles of the record and query reversed. Records are selected based on how well they match some piece of the query, with no penalty for unmatched sections of the query. Use this score type to categorize documents by using the document as a query against a table consisting of records of known keywords.

Minimum ( pass in NetricsSearchOpts.MINIMUM). Minimum of normal and reverse scores.

Maximum ( pass in NetricsSearchOpts.MAXIMUM). Maximum of normal and reverse scores.

Scoreit ( pass in NetricsSearchOpts.SCORE_IT). This is a measure of informational difference between query and record somewhat similar to the symmetric score. It is very rarely used.

All six score types are computed and returned for each record and can be accessed using the getNormMatchScore, getRevMatchScore, getSymMatchScore, getMinMatchScore, getMaxMatchScore and getITMatchScore NetricsSearchResult methods. The getMatchScore method holds a copy of whichever match score was used to select and sort the records in the list.