# TIBCO Patterns - Search Query Builder Platform

# TIBCO Patterns - Search Query Builder Platform Overview

The TIBCO Patterns - Search Query Builder Platform is a Java based tool that makes it easier to design, test and administer TIBCO Patterns - Search record matching queries. It eliminates or greatly reduces the need to write Java code to define and run record matching queries. It replaces Java code, which must be compiled and linked into an application, with an external configuration file, which can be loaded at run time.

The Query Builder Platform provides an XML schema for query configuration files, Java classes to process the configurations, and tools to validate and test the query configurations.

## TIBCO Patterns - Search Query Builders

A TIBCO Patterns - Search query performs a search for records matching a particular set of values. To search for "John Smith" you build a query and perform the search. To search for "Jack Jones" you must build a new query. Every new search requires a new query be built. A query builder is something that takes in a set of values to be matched and outputs a TIBCO Patterns - Search query to match those values.

The TIBCO Patterns - Search Java API provides two abstract classes that define a standard interface for query builders. `ANetricsQueryBuilder` builds queries to match a flat record. `ANetricsCompoundQueryBuilder` builds queries to match a compound record (see the TIBCO Patterns - Search Concepts Guide chapter on Joins, section on Compound Records for an explanation of compound records). It is a good practice to implement your query builder as an extension of one of these classes, and to use the abstract classes in your application. This allows queries to be portable across applications.

These two classes provide methods that enable an implementation of a query to be adapted to new applications without code changes. Input and output data element names can be remapped, as well as table names. The classes accept a number of input formats, taking care of the bookkeeping needed to deal with the different formats. The implementer only needs to define how to build the query, the rest of the functionality is provided by the abstract base class. This simplifies the writing of a new query builder.

## Query Builder Platform Overview

The classes `ANetricsQueryBuilder` and `ANetricsCompoundQueryBuilder` provide a standard interface for query builders, making them portable across applications. They also simplify the process of writing a query builder by taking care of routine bookkeeping operations. But you must still write the code, compile it and link it into your application. The Query Builder Platform eliminates the need to write code. Instead a configuration file is used to define the query. From the configuration file the platform can either generate the code for you, or build queries directly from the configuration file.

Details on the configuration file used to define a query can be found in the QueryDef.xsd file in the query_builder schema directory. There is also a sample configuration file, customer1.xml, in the samples directory.

The Query Builder Platform provides extensions of the query builder abstract classes in the Java API; `GeneralQueryBuilder` extends `ANetricsQueryBuilder` and `GeneralCompoundQueryBuilder` extends `ANetricsCompoundQueryBuilder`. `GeneralQueryBuilder` and `GeneralCompoundQueryBuilder` read a configuration file that completely defines how to build the query. Because `GeneralQueryBuilder` and `GeneralCompoundQueryBuilder` are extensions of the abstract classes, they can be plugged into any application designed to use one of the abstract classes as its query builder. The query used by the application is then defined from a configuration file read in at run time, rather than from compiled code that must be linked into the application.

The platform provides additional functionality. A tool is provided that allows query configurations to be verified and tested. A set of interfaces and abstract classes provide hooks that allow `GeneralCompoundQueryBuilder` and `GeneralQueryBuilder` to be integrated into almost any application.

## Using the Query Builder Platform

The Query Builder Platform provides a command line interface for testing a query definition. There are two parts to testing a query definition: verifying that the configuration file is valid and verifying that the query retrieves the desired records. You can also use the platform to generate Java code based on a query definition in a configuration file. The platform can be embedded into other applications, allowing the the TIBCO Patterns - Search queries used by the application to be defined by configuration files.

### Verifying the configuration

This step verifies that the configuration file defining the query is valid and generates a valid TIBCO Patterns - Search query. The command line tool scans the configuration producing a log of all errors and questionable items found in the configuration. On request it creates a copy of the configuration file with error messages added as annotations to the associated items in the configuration. This is often the best way to locate where the errors are in a large and complex configuration.

### Verifying the query

Verifying that a configuration is valid does not verify that the query it defines returns the desired results. To do that the query must be run and the results of the query examined. The command line interface allows you to run a query against a TIBCO Patterns - Search server using data from CSV files or a TIBCO Patterns - Search server.

The running of queries is defined through a configuration file. This is a separate configuration file from the one that defines the query. This file lets you specify the source data for generating the queries and a target TIBCO Patterns - Search server, on which the queries are run.

The query source can be a set of CSV files, one for each table defined for the query. The rows in the files must be sorted by the PARENT key value. The records in the CSV files are merged into compound records. These compound records are used as the input to the defined query builder.

Alternatively, the query source can be a TIBCO Patterns - Search server. Compound records are read from the server based on the table structure in the query definition.

The query run configuration file also lets you remap the table and field names for the query source and the target server. It also lets you specify a cutoff score applied to all queries, and a maximum number of queries to run.

Details on the query run configuration file can be found in the RunQuery.xsd file in the query_builder schema directory. There is also a sample query run configuration file, run_customer1.xml, in the samples directory.

### Generating Code

You can also use the query builder platform to generate Java source that implements the query builder defined by the configuration file. The generated Java source extends and uses classes contained in the query builder platform JAR file (TIB_tps_qbp.jar). Therefore, you must include the TIB_tps_qbp.jar file in the class path of any application that uses the generated query builder classes. If that file is included not from its installed location, you must also include the JAXB JAR files installed in the external\jaxb\mod directory. However, the full configuration file is not needed at run time.

- For generated single table record matching queries, a configuration file is not required at run time. The exception is when a custom query builder class requires a configuration file.
- For compound record matching queries a run time configuration file is required that defines the options for the compound query builder. The tool generates this configuration.

In most cases, no practical advantage is realized by using the generated code. It may run slightly faster, however in most cases the performance difference is not noticeable. Avoiding the need for loading configurations may be an advantage in certain applications where a file system or other data store is not available. The primary use for generating the Java source is where manual editing of the source is necessary to provide special functionality. The general query structure can be generated from a configuration and then customized manually.

### Using the Query Builder Platform in an Application

Because of its flexibility in allowing queries to be changed quickly and easily, most applications using the Java API should use the query builder platform to implement their queries. Production experience, or changes in requirements, often require modifications to the query being used. It is much easier to update a configuration file, than to install new code.

The two classes, `GeneralCompoundQueryBuilder` and `GeneralQueryBuilder`, are extensions of `ANetricsCompoundQueryBuilder` and `ANetricsQueryBuilder`. Therefore, they can be used in any Java application that expects an implementation of one of these classes. The `GeneralCompoundQueryBuilder` or `GeneralQueryBuilder` class must be initialized with the desired configuration, either in the constructor when created, or by calling one of the `setConfiguration` methods before being used. To use these classes the application must include the TIB_tps_qbp.jar file in its class path. If that file is included not from its installed location, the application must also include the JAXB JAR files installed in the external\jaxb\mod directory.

Some applications are required to handle either compound record matching or single table record matching. In such cases, the application can use `GeneralCompoundQueryBuilder`. Standard, single table, record matching is provided as one of the compound record matching types available in the query configuration.

To allow the `GeneralCompoundQueryBuilder` and `GeneralQueryBuilder` classes to fit seamlessly into almost any application, three interfaces are defined by the query builder platform: `AQBPLogger`, `IQBPFileMgr`, and `IQBPLoaderMgr`. Default implementations of these interfaces are applied automatically if the application does not supply its own. Therefore, it is not necessary to provide an implementation of any of these classes. An implementation is needed only if the default behavior is not suitable.

#### AQBPLogger

Gives the application control over the reporting of errors.

The application can provide its own implementation of this abstract class to integrate the logging and error reporting of the query builder platform classes with the application's logging or the application can use the provided `QBPFileLogger` class and set options to control the reporting of errors. When using the `QBPFileLogger` class, you can set options to perform the following actions:

- throw exceptions for errors rather than log them
- add or not add error annotations to the configuration file
- control the types of log entries reported.

#### IQBPFileMgr

Gives the application control over the reading of configuration files.

This can include enforcing restrictions on which files can be read. It can also include allowing configurations to be read from non-standard locations, such as a DBMS or a web-service.

A basic implementation of this interface, `DfltFileMgr`, is provided. It reads configuration files from a standard file system. By default, it imposes no restrictions, but it can be set to restrict configuration files to a specific directory (and all sub-directories).

#### IQBPLoaderMgr

Gives the application control over the dynamic loading of classes.

If the configuration file contains custom classes, these classes must be dynamically loaded. For security reasons, by default, classes are only loaded from the JVM class path. Applications may want to allow classes to be loaded from other locations (such as a directory for custom class files). Applications can define their own rules for the dynamic loading of classes by providing an appropriate implementation of this interface. As with the loading of configurations, this can also be used to allow custom classes to be loaded from non-standard sources, such as a DBMS or web-service.

A basic implementation of this interface, `DfltLoaderMgr`, is provided. By default the `DfltLoaderMgr` loads classes only from the JVM class path. It can be set to load classes from a specific directory in addition to the JVM class path.

### Integration with Other TIBCO Patterns - Search Tools

The TIBCO Patterns - Search deduplication framework provides two sample implementations of its `QueryBuilder` interface. `GenericCompoundQueryBuilder` wraps an `ANetricsCompoundQueryBuilder` object as an implementation of the deduplication framework `QueryBuilder` interface. `GenericQueryBuilder` wraps `ANetricsQueryBuilder` in a similar fashion. The deduplication framework sample project shows how these classes can be used, in combination with the Query Builder Platform `GeneralCompoundQueryBuilder` class, to create a deduplication project that is driven from a query definition configuration file instead of from Java code.

## Query Builder Platform Package Contents

The Query Builder Platform is installed under the query_builder directory in the TIBCO Patterns - Search home directory. Under this directory are the following directories:

### schema

This directory contains three XSD files. They contain detailed comments; they can be used as a reference to the format of the configuration files used in the query builder platform.

- QueryDef.xsd is the schema for the query definition configuration file.
- PatternsSchema.xsd defines the basic TIBCO Patterns - Search query types used in the QueryDef.xsd file. It is not used directly.
- RunQuery.xsd is the schema for the query run configuration file.

### sample

This directory contains a sample query builder project. It contains a sample query definition configuration file and a sample query run configuration file. Both are extensively commented. It also contains sample data, a load script and a README.txt file that gives detailed instructions for running the sample project.

### bin

This contains an executable jar file that is the command line interface for the Query Builder Platform. Executing `java -jar bin/TIB_tps_qbpcmd.jar -help` outputs a help message describing the arguments to this command. This help message can also be found in this API documentation under the main method of the `GeneralCompoundQueryBuilder` class.

### lib

This contains the run time java library (TIB_tps_qbp.jar) for the Query Builder Platform.

Packages
Package | Description
--- | ---
com.tibco.patterns.qbp |