Package com.tibco.patterns.learn.api.project

## Class ModelTrainer

- java.lang.Object
- - com.tibco.patterns.learn.api.project.ModelTrainer

-     public final class ModelTrainer
      extends java.lang.Object

  Functionality to communicate with Patterns server, load data table, calculate feature scores, train and evaluate model.

  A typical usage sequence:
  1. Create ModelTrainer. Server must be running.
  2. Provide table or table definition:
  a. Use table on the server. Use tableExists(), then loadTable(), reloadTable() or setTable().
  b. Use setTableDef() to score pairs without having a table on server. If data file changes, close and recreate ModelTrainer.
  3. calcAllFeatureScores() - creates scorer.
  4. createExperiment()
  5. Train the model:
  a. Automatic training until convergence - use trainToConverge(). Or:
  b. One iteration at a time - use beginTraining(), trainIteration(). If `COErrorRateMin` is used, retrain the model after the first training determines the best number of iterations.
  6. After adding/changing record pairs (but not features), repeat items 4, 5 to train a new model.
  7. If the features have changed, repeat items 3, 4, 5.
  8. close() - must be called at the end if scorer was created or model was created/loaded.

- - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `void` | `beginTraining​(ConvergenceObserver trainCO, ConvergenceObserver vldCO)` | Creates model using the model configuration from modelTrainConfig.xml file, or the default configuration if that file is not in the project directory.
    `void` | `beginTraining​(ConvergenceObserver trainCO, ConvergenceObserver vldCO, ModelConfig modelConfig)` | Creates model and prepares training structures to start step-by-step training.
    `void` | `calcAllFeatureScores()` | Creates a new record pair scorer and recalculates feature scores for pairs in datasets used for model creation and training in ModelSettings.
    `void` | `calcFeatureScores​(RecPair recPair)` | Calculates symmetric feature scores and saves them in the given pair.
    `void` | `calcFeatureScores​(java.util.List<RecPair> recPairs)` | Calculates symmetric feature scores for each pair and saves them in the given RecPair objects.
    `void` | `calcMissingFeatureScores()` | Calculates any missing feature scores in datasets used for model creation and training, stores them in ModelSettings.
    `com.netrics.likeit.NetricsServerInterface.ShutdownStatus` | `close​(boolean shutDownServer)` | Closes scorer if it was created - deletes thesauri and model (if loaded) from server.
    `void` | `createExperiment()` | Calculates any missing feature scores in datasets used for model creation and training, saves them to ModelSettings.
    `java.util.Map<Partition,​ConvergenceObserver>` | `createObservers()` | Creates convergence observers from modelTrainConfig.xml file, or creates default observers (COErrorRate and COErrorRateMin) if that file is not in the project directory.
    `int` | `datasetSize​(DataPartition partition)` |
    `void` | `deleteTable()` | Deletes the loaded table from server, if it is found on the server.
    `void` | `evaluate​(DataPartition partition, TrainingObserver co)` | Evaluates the existing model using examples from specified partition in the existing experiment.
    `void` | `evaluate​(ModelSettings pairSource, DataPartition partition, TrainingObserver co)` | Evaluates the existing model with the specified dataset loaded from another ModelSettings object (pairSource).
    `void` | `evaluateAll​(ModelSettings pairSource, TrainingObserver co)` | Evaluates the existing model with record pairs in datasets used for training loaded from another ModelSettings object (pairSource).
    `void` | `evaluateAll​(TrainingObserver co)` | Evaluates the existing model using examples from both datasets in the existing experiment.
    `void` | `evaluateCsv​(java.lang.String csvFileName, TrainingObserver co)` | Loads record pairs from the CSV file and evaluates them.
    `protected void` | `finalize()` | Provided for safety, but client code must call close() explicitly.
    `com.netrics.likeit.NetricsRecord[]` | `firstRecBatch()` | Prepares the server table for iteration over all records.
    `com.netrics.likeit.NetricsRecord[]` | `firstRecBatch​(int batchSize, java.lang.String startPos)` | Prepares the server table for iteration over records using specific options.
    `java.util.List<com.netrics.likeit.NetricsSearchResult>` | `getLowConfMatches​(java.util.List<java.lang.String> queryValues, double confThreshold)` | Runs a query on server to get the low confidence matches in the data table for the given query record.
    `PredictOptions` | `getPredictOptions()` |
    `java.lang.String` | `getTablePosition()` | Gets the encoded current position in the table that is used to get the next batch of records.
    `boolean` | `hasModel()` |
    `boolean` | `isModelLoadedToServer()` | Checks if the model was loaded to server by the scorer.
    `boolean` | `isRandomOrder()` | Get the random order parameter that was specified in ModelTrainConfig (if file present), or the default (true).
    `void` | `loadTable​(java.lang.String tableName, com.netrics.likeit.NetricsRecFile csvFile)` | Loads the table from a server side CSV file to the Patterns server for scoring.
    `com.netrics.likeit.NetricsRecord[]` | `nextRecBatch()` | Gets the next batch of records from the server table.
    `void` | `predict​(RecPair recPair)` | Predicts the given pair using the existing model.
    `void` | `reloadTable​(java.lang.String tableName, com.netrics.likeit.NetricsRecFile csvFile)` | Deletes table from the server if it exists, then loads table from a server side CSV file.
    `void` | `setModelTrainConfig​(ModelTrainConfig mtConfig)` | Remembers the ModelTrainConfig reference (can be further modified from outside).
    `void` | `setPredictOptions​(PredictOptions predictOpts)` | Sets options to be used for all subsequent model predictions using this ModelTrainer.
    `void` | `setTable​(java.lang.String tableName)` | Sets the name of the table that already exists on the server.
    `void` | `setTableDef()` | Sets the table definition for pair scoring.
    `boolean` | `tableExists​(java.lang.String tableName)` |
    `boolean` | `tableIsCompatible​(java.lang.String tableName)` | Checks if the table on server has field names and types that match ModelSettings.
    `java.lang.String` | `toString()` |
    `boolean` | `trainIteration()` | Performs a single training iteration.
    `boolean` | `trainToConverge​(ConvergenceObserver trainCO, ConvergenceObserver vldCO)` | Trains the model using the model configuration from modelTrainConfig.xml file, or the default configuration if that file is not in the project directory.
    `boolean` | `trainToConverge​(ConvergenceObserver trainCO, ConvergenceObserver vldCO, ModelConfig mConfig)` | Creates and trains model with the prepared experiment data until stopping criteria are satisfied.
    `void` | `updatePairsFromTable()` | Updates field data in pairs from the (modified) data table.
    `void` | `verifyHasModel()` |
    `void` | `verifyServerTableLoaded()` | Verifies that the server table (not a table definition) was assigned, and that this table currently exists on the server.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Method Detail

    - #### toString

          public java.lang.String toString()

      Overrides:
        `toString` in class `java.lang.Object`

    - #### setModelTrainConfig

          public void setModelTrainConfig​(ModelTrainConfig mtConfig)

      Remembers the ModelTrainConfig reference (can be further modified from outside). Called when a new ModelTrainConfig is created. Can also be called before training to overwrite the ModelTrainConfig specified by ProjectFiles with a custom configuration.

    - #### tableExists

          public boolean tableExists​(java.lang.String tableName)
                              throws com.netrics.likeit.NetricsException,
                                     java.io.IOException

      Returns:
        true if table with the specified name exists on the Patterns server
      Throws:
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### verifyServerTableLoaded

          public void verifyServerTableLoaded()
                                       throws com.netrics.likeit.NetricsException,
                                              java.io.IOException

      Verifies that the server table (not a table definition) was assigned, and that this table currently exists on the server.
      Throws:
        `java.lang.IllegalStateException` - if the server table has not been assigned/loaded.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### tableIsCompatible

          public boolean tableIsCompatible​(java.lang.String tableName)
                                    throws com.netrics.likeit.NetricsException,
                                           java.io.IOException

      Checks if the table on server has field names and types that match ModelSettings.
      Parameters:
        `tableName` - - the name of the table on server
      Returns:
        true if the table on server has field names and types that match ModelSettings
      Throws:
        `java.lang.IllegalStateException` - if key field is not assigned.
        `com.netrics.likeit.NetricsException` - if table is not found on server, or other error on server
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### setTable

          public void setTable​(java.lang.String tableName)
                        throws com.netrics.likeit.NetricsException,
                               java.io.IOException

      Sets the name of the table that already exists on the server.
      Parameters:
        `tableName` - - the name of the table on server
      Throws:
        `java.lang.IllegalArgumentException` - if the specified table does not exist on server, or it is found on server but has different number of fields or different field types.
        `java.lang.IllegalStateException` - if the server table or table definition is already assigned; or if the number, names or types of fields on server table do not match info in ModelSettings, or if key field is not assigned.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### loadTable

          public void loadTable​(java.lang.String tableName,
                                com.netrics.likeit.NetricsRecFile csvFile)
                         throws com.netrics.likeit.NetricsException,
                                java.io.IOException

      Loads the table from a server side CSV file to the Patterns server for scoring. Uses field types from ModelSettings. May take a long time for large tables.
      Parameters:
        `tableName` - - name of the new table on the server (must not exist)
        `csvFile` - - defines format of the server side CSV file to be loaded.
      Throws:
        `java.lang.IllegalArgumentException` - if table with the specified name already exists on server.
        `java.lang.IllegalStateException` - if server table or table definition is already assigned.
        `com.netrics.likeit.NetricsException` - - error on server. RECEXISTS - duplicates in key field.
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### reloadTable

          public void reloadTable​(java.lang.String tableName,
                                  com.netrics.likeit.NetricsRecFile csvFile)
                           throws com.netrics.likeit.NetricsException,
                                  java.io.IOException

      Deletes table from the server if it exists, then loads table from a server side CSV file. Allows reloading an already assigned table with the same name.
      Throws:
        `java.lang.IllegalStateException` - if table definition is already assigned.
        `java.lang.IllegalArgumentException` - if tableName is different from the name of already assigned table.
        `com.netrics.likeit.NetricsException` - - error on server. RECEXISTS - duplicates in key field.
        `java.io.IOException` - if an I/O error occurs while communicating with server
      See Also:
        `loadTable(String, NetricsRecFile)`

    - #### deleteTable

          public void deleteTable()
                           throws com.netrics.likeit.NetricsException,
                                  java.io.IOException

      Deletes the loaded table from server, if it is found on the server. Optional operation. Any created scorer and models become invalid and are destroyed. After this call, another table could be loaded or a table definition assigned (but it is simpler to create a new ModelTrainer).
      Throws:
        `java.lang.IllegalStateException` - - if no table was assigned/loaded.
        `com.netrics.likeit.NetricsException` - - If the server indicates that an error has occured
        `java.io.IOException` - - If an I/O error occurs while communicating with server

    - #### setTableDef

          public void setTableDef()

      Sets the table definition for pair scoring. Table definition is obtained from the associated ModelSettings. This allows scoring record pairs without uploading the table to server. Methods that retrieve field values are not supported without having the table on server. After table definition is assigned, server table cannot be assigned.
      Throws:
        `java.lang.IllegalStateException` - if server table or table definition is already assigned, or if key field is not assigned.

    - #### firstRecBatch

          public com.netrics.likeit.NetricsRecord[] firstRecBatch​(int batchSize,
                                                                  java.lang.String startPos)
                                                           throws com.netrics.likeit.NetricsException,
                                                                  java.io.IOException

      Prepares the server table for iteration over records using specific options.
      Parameters:
        `batchSize` - - the size of each record batch that is returned.
        `startPos` - - the initial position. If null or the position is at the end of table, starts at the first record (by ID).
      Returns:
        the first batch of records in the server table, or null if table is empty.
      Throws:
        `java.lang.IllegalStateException` - if server table is not assigned to this ModelTrainer.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if I/O error occurs while communicating with server

    - #### firstRecBatch

          public com.netrics.likeit.NetricsRecord[] firstRecBatch()
                                                           throws com.netrics.likeit.NetricsException,
                                                                  java.io.IOException

      Prepares the server table for iteration over all records.
      Returns:
        the first batch of records in the server table, or null if table is empty.
      Throws:
        `java.lang.IllegalStateException` - if server table is not assigned to this ModelTrainer.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if I/O error occurs while communicating with server

    - #### nextRecBatch

          public com.netrics.likeit.NetricsRecord[] nextRecBatch()
                                                          throws com.netrics.likeit.NetricsException,
                                                                 java.io.IOException

      Gets the next batch of records from the server table.
      Returns:
        the next batch of 1000 records in the server table, or null at end of table.
      Throws:
        `java.lang.IllegalStateException` - if server table does not exist, or if `firstRecBatch()` was not called for the current server table.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if I/O error occurs while communicating with server

    - #### getTablePosition

          public java.lang.String getTablePosition()

      Gets the encoded current position in the table that is used to get the next batch of records.
      Returns:
        the current position in the table.
      Throws:
        `java.lang.IllegalStateException` - if `firstRecBatch()` was not called for the current server table.

    - #### updatePairsFromTable

          public void updatePairsFromTable()
                                    throws com.netrics.likeit.NetricsException,
                                           java.io.IOException

      Updates field data in pairs from the (modified) data table. Only pairs in the active model settings are updated (method should not be used if other saved models exist). If a documented exception is thrown, no pairs are modified.
      Throws:
        `java.lang.IllegalStateException` - if server table does not exist, or if any pair record is not found in the table, or if the number of fields in table is invalid.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured.
        `java.io.IOException` - if I/O error occurs while communicating with server.

    - #### calcAllFeatureScores

          public void calcAllFeatureScores()
                                    throws com.netrics.likeit.NetricsException,
                                           java.io.IOException

      Creates a new record pair scorer and recalculates feature scores for pairs in datasets used for model creation and training in ModelSettings. Feature scores depend only on the non-key field values stored in the pair. Any previous scorer and trained models are discarded. ModelSettings must contain non-empty feature query. Must be called before using calcMissingFeatureScores() or createExperiment(). Must be called again before using these methods if there were any changes to the feature query.
      Throws:
        `java.lang.IllegalStateException` - if neither the server table nor table definition is assigned, or if the server table is assigned but not loaded, or if the feature query is empty.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### calcMissingFeatureScores

          public void calcMissingFeatureScores()
                                        throws com.netrics.likeit.NetricsException,
                                               java.io.IOException

      Calculates any missing feature scores in datasets used for model creation and training, stores them in ModelSettings. Feature scores depend only on the non-key field values stored in the pair. Any previously trained models are discarded. Saves time by using the existing scorer and not recalculating existing feature scores. Scorer must be already created and the feature query must not have changed since then (client code is responsible for tracking such changes). ModelSettings must contain unchanged feature query.
      Throws:
        `java.lang.IllegalStateException` - if the scorer has not been created, or if the server table is assigned but not loaded, or if ms does not contain training/validation record pairs labeled true/false, or if no features exist.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### calcFeatureScores

          public void calcFeatureScores​(java.util.List<RecPair> recPairs)
                                 throws com.netrics.likeit.NetricsException,
                                        java.io.IOException

      Calculates symmetric feature scores for each pair and saves them in the given RecPair objects. Does not modify the project. Always (re)creates scorer.
      Parameters:
        `recPairs` - = record pairs to be scored.
      Throws:
        `java.lang.IllegalStateException` - if neither the server table nor table definition is assigned, or if the server table is assigned but not loaded, or if no features exist.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### calcFeatureScores

          public void calcFeatureScores​(RecPair recPair)
                                 throws com.netrics.likeit.NetricsException,
                                        java.io.IOException

      Calculates symmetric feature scores and saves them in the given pair. Uses existing scorer, or creates scorer if it has not been created yet.
      Parameters:
        `recPair` - = record pair to be scored.
      Throws:
        `java.lang.IllegalStateException` - if the server table is assigned but not loaded, or if no features exist.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### createExperiment

          public void createExperiment()
                                throws com.netrics.likeit.NetricsException,
                                       java.io.IOException

      Calculates any missing feature scores in datasets used for model creation and training, saves them to ModelSettings. Creates experiment with boolean-labeled pairs from datasets in ModelSettings. Any previously trained and unsaved model is discarded. Saves time by using the existing scorer and not recalculating existing feature scores. Scorer must be already created and the feature query must not have changed since then (client code is responsible for tracking such changes).
      Throws:
        `java.lang.IllegalStateException` - - if the scorer has not been created, or if the server table is assigned but not loaded, or if no features exist.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### datasetSize

          public int datasetSize​(DataPartition partition)

      Returns:
        number of examples in the given dataset of the experiment. Returns 0 if the given dataset does not participate in training.
      Throws:
        `java.lang.IllegalStateException` - if experiment was not created

    - #### isRandomOrder

          public boolean isRandomOrder()

      Get the random order parameter that was specified in ModelTrainConfig (if file present), or the default (true).
      Returns:
        true if the order of training examples is randomized before every iteration.

    - #### setPredictOptions

          public void setPredictOptions​(PredictOptions predictOpts)

      Sets options to be used for all subsequent model predictions using this ModelTrainer. If specific options object is not set, then default options are used.
      Parameters:
        `predictOpts` - the prediction options. If null, default options will be used.

    - #### getPredictOptions

          public PredictOptions getPredictOptions()

      Returns:
        the options that are currently used for all model predictions. Returns null if default options are used.

    - #### createObservers

          public java.util.Map<Partition,​ConvergenceObserver> createObservers()

      Creates convergence observers from modelTrainConfig.xml file, or creates default observers (COErrorRate and COErrorRateMin) if that file is not in the project directory.
      Returns:
        the created observers for Training and Validation datasets.

    - #### beginTraining

          public void beginTraining​(ConvergenceObserver trainCO,
                                    ConvergenceObserver vldCO)

      Creates model using the model configuration from modelTrainConfig.xml file, or the default configuration if that file is not in the project directory. See `beginTraining(ConvergenceObserver, ConvergenceObserver, ModelConfig)`
      Throws:
        `java.lang.NullPointerException` - if either of the required parameters is null
        `java.lang.IllegalArgumentException` - if trainCO or vldCO has been used in previous training, or if the same object is used as trainCO and vldCO, or if vldCO is COErrorRateMin and the same trainCO was not used when creating vldCO.
        `java.lang.IllegalStateException` - if experiment was not created.

    - #### beginTraining

          public void beginTraining​(ConvergenceObserver trainCO,
                                    ConvergenceObserver vldCO,
                                    ModelConfig modelConfig)

      Creates model and prepares training structures to start step-by-step training. Destroys any previous model. The created model must be trained for at least one iteration before it can be saved.
      Parameters:
        `trainCO` - - training observer for training set. Not null. Must be COErrorRate or its subclass, else the trained model cannot be saved.
        `vldCO` - - convergence observer for validation set. Not null. Must be COErrorRate or its subclass, else the trained model cannot be saved.
        `modelConfig` - - model configuration to create the model. Not null. The number of features set in this object is ignored. This method assigns false subsets.
      Throws:
        `java.lang.NullPointerException` - if either of the required parameters is null.
        `java.lang.IllegalArgumentException` - if trainCO or vldCO has been used in previous training, or if the same object is used as trainCO and vldCO, or if vldCO is COErrorRateMin and the same trainCO was not used when creating vldCO, or if the number of features in the dynamic parameter arrays is invalid.
        `java.lang.IllegalStateException` - if experiment was not created

    - #### hasModel

          public boolean hasModel()

      Returns:
        true if an RLink model has been created or loaded to ModelTrainer.

    - #### verifyHasModel

          public void verifyHasModel()

      Throws:
        `java.lang.IllegalStateException` - if the model was not created or loaded.

    - #### trainIteration

          public boolean trainIteration()

      Performs a single training iteration. Call beginTraining() before training any iterations. Updates statistics in observers that were passed to beginTraining().
      Returns:
        true if the training should continue after this iteration (as indicated by training and validation observers).
      Throws:
        `java.lang.IllegalStateException` - if experiment was not created, or if model was not created or loaded, or if training was not started.

    - #### trainToConverge

          public boolean trainToConverge​(ConvergenceObserver trainCO,
                                         ConvergenceObserver vldCO)

      Trains the model using the model configuration from modelTrainConfig.xml file, or the default configuration if that file is not in the project directory. See `trainToConverge(ConvergenceObserver, ConvergenceObserver, ModelConfig)`
      Throws:
        `java.lang.IllegalArgumentException` - if trainCO or vldCO has been used in previous training, or if the same object is used as trainCO and vldCO, or if vldCO is COErrorRateMin and the same trainCO was not used when creating vldCO.
        `java.lang.IllegalStateException` - if experiment was not created.

    - #### trainToConverge

          public boolean trainToConverge​(ConvergenceObserver trainCO,
                                         ConvergenceObserver vldCO,
                                         ModelConfig mConfig)

      Creates and trains model with the prepared experiment data until stopping criteria are satisfied. Destroys any previous model.
      Parameters:
        `trainCO` - - training observer for training set. Must be COErrorRate or its subclass, else the trained model cannot be saved.
        `vldCO` - - convergence observer for validation set. Not null. If this is COErrorRateMin, then this method trains the model twice, using the determined best number of iterations for the second training. Must be COErrorRate or its subclass, else the trained model cannot be saved.
        `mConfig` - - model configuration to create the model. Not null. The number of features set in this object is ignored.
      Returns:
        true if the (first) model training has converged.
      Throws:
        `java.lang.IllegalArgumentException` - if trainCO or vldCO has been used in previous training, or if the same object is used as trainCO and vldCO, or if vldCO is COErrorRateMin and the same trainCO was not used when creating vldCO, or if the number of features in the dynamic parameter arrays of mConfig is invalid.
        `java.lang.IllegalStateException` - if experiment was not created

    - #### evaluate

          public void evaluate​(DataPartition partition,
                               TrainingObserver co)

      Evaluates the existing model using examples from specified partition in the existing experiment.
      Parameters:
        `partition` - - the partition to evaluate. Must be TRAIN or VLD.
        `co` - - training observer to accumulate test results for the given dataset. Not null.
      Throws:
        `java.lang.IllegalArgumentException` - if co has been used in previous training, or if the given partition is not used in model training.
        `java.lang.IllegalStateException` - if experiment was not created, or if model was not created or loaded.

    - #### evaluateAll

          public void evaluateAll​(TrainingObserver co)

      Evaluates the existing model using examples from both datasets in the existing experiment.
      Parameters:
        `co` - - training observer to accumulate test results for the given dataset. Not null.
      Throws:
        `java.lang.IllegalArgumentException` - if co has been used in previous training.
        `java.lang.IllegalStateException` - if experiment was not created, or if model was not created or loaded.

    - #### evaluate

          public void evaluate​(ModelSettings pairSource,
                               DataPartition partition,
                               TrainingObserver co)
                        throws com.netrics.likeit.NetricsException,
                               java.io.IOException

      Evaluates the existing model with the specified dataset loaded from another ModelSettings object (pairSource). Creates scorer. Does not save predictions to dataset.
      Parameters:
        `co` - - training observer to accumulate test results for the given dataset. Not null.
      Throws:
        `java.lang.IllegalArgumentException` - if co has been used in previous training.
        `java.lang.IllegalStateException` - if model was not created or loaded, or if neither the server table nor table definition is assigned, or if server table is assigned but not loaded, or if no features are present.
        `java.io.IOException` - if I/O error occurs while communicating with server
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured

    - #### evaluateAll

          public void evaluateAll​(ModelSettings pairSource,
                                  TrainingObserver co)
                           throws com.netrics.likeit.NetricsException,
                                  java.io.IOException

      Evaluates the existing model with record pairs in datasets used for training loaded from another ModelSettings object (pairSource). Creates scorer. Does not save predictions to dataset.
      Parameters:
        `co` - - training observer to accumulate test results for the given dataset. Not null.
      Throws:
        `java.lang.IllegalArgumentException` - if co has been used in previous training.
        `java.lang.IllegalStateException` - if model was not created or loaded, or if neither the server table nor table definition is assigned, or if server table is assigned but not loaded, or if no features are present.
        `java.io.IOException` - if I/O error occurs while communicating with server
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured

    - #### evaluateCsv

          public void evaluateCsv​(java.lang.String csvFileName,
                                  TrainingObserver co)
                           throws com.netrics.likeit.NetricsFileFormatException,
                                  java.io.IOException,
                                  com.netrics.likeit.NetricsException

      Loads record pairs from the CSV file and evaluates them. Allows duplicate pairs in CSV. Always creates scorer. Requires the table on server.
      Parameters:
        `co` - - training observer to accumulate test results for the CSV dataset.
      Throws:
        `java.lang.IllegalStateException` - if the server table has not been assigned/loaded, or if model was not created or loaded.
        `java.lang.IllegalArgumentException` - if number of query field values does not match the number of all fields in feature query.
        `java.io.FileNotFoundException` - if the CSV file is not found
        `com.netrics.likeit.NetricsFileFormatException` - if CSV file format is incorrect
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if pairs CSV file has wrong number of entries per line; or I/O error occurs while communicating with server

    - #### predict

          public void predict​(RecPair recPair)
                       throws com.netrics.likeit.NetricsException,
                              java.io.IOException

      Predicts the given pair using the existing model. Calculates and stores feature scores and the model prediction in the given pair. Uses existing scorer or creates it if not created yet. Does not modify the project.
      Parameters:
        `recPair` - - the pair used to calculate model prediction. Label is not required.
      Throws:
        `java.lang.IllegalStateException` - if model was not created or loaded, or if a boolean label is not assigned to the given RecPair.
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### isModelLoadedToServer

          public boolean isModelLoadedToServer()

      Checks if the model was loaded to server by the scorer. It does not check if the loaded model still exists on the server.
      Returns:
        true if the trained model has been leaded to server.

    - #### getLowConfMatches

          public java.util.List<com.netrics.likeit.NetricsSearchResult> getLowConfMatches​(java.util.List<java.lang.String> queryValues,
                                                                                          double confThreshold)
                                                                                   throws com.netrics.likeit.NetricsException,
                                                                                          java.io.IOException

      Runs a query on server to get the low confidence matches in the data table for the given query record. The query uses the model and thesauri on server.
      Parameters:
        `queryValues` - - field values for the query record. Not null.
        `confThreshold` - - return records with confidence less than or equal to this threshold.
      Returns:
        the low confidence matches in the data table returned by the query.
      Throws:
        `com.netrics.likeit.NetricsException` - if the server indicates that an error has occured
        `java.io.IOException` - if an I/O error occurs
        `java.lang.IllegalStateException` - if the scorer has not been created, or if the data table or the model was not loaded to server, or if feature confidence is not used.

    - #### close

          public com.netrics.likeit.NetricsServerInterface.ShutdownStatus close​(boolean shutDownServer)
                                                                         throws com.netrics.likeit.NetricsException,
                                                                                java.io.IOException

      Closes scorer if it was created - deletes thesauri and model (if loaded) from server. Destroys any created or loaded RLink model. Optionally shuts down the server. Must be called after the scorer has been created (by calling `calcAllFeatureScores()`) or an RLink model has been created/loaded and the object will no longer be used. After this call, if the server has not been shut down, the object is still usable: any created experiment still exists; client can create a new scorer and experiment and/or train a new model.
      Parameters:
        `shutDownServer` - - if true also shuts down the running server with 15-second timeout.
      Returns:
        shutdown status if shutDownServer is true (INITIATED means timeout expired and the server is still shutting down); null if shutDownServer is false.
      Throws:
        `com.netrics.likeit.NetricsException` - if a created object no longer exists on the server or other server error has occured
        `java.io.IOException` - if an I/O error occurs while communicating with server

    - #### finalize

          protected void finalize()
                           throws com.netrics.likeit.NetricsException,
                                  java.io.IOException

      Provided for safety, but client code must call close() explicitly.
      Overrides:
        `finalize` in class `java.lang.Object`
      Throws:
        `com.netrics.likeit.NetricsException`
        `java.io.IOException`