Package com.tibco.patterns.learn.training

## Class COErrorRate

- java.lang.Object
- - com.tibco.patterns.learn.training.COIterationCount
  - - com.tibco.patterns.learn.training.COErrorRate

- All Implemented Interfaces:
    `ConvergenceObserver`, `TrainingObserver`
  Direct Known Subclasses:
    `COErrorRateMin`

      public class COErrorRate
      extends COIterationCount

  Stops the training when the error rate over validation dataset during the last iteration is below the given threshold. Gathers additional statistics. The error rate and other statistics include predictions made by untrained submodels. This class and its descendants can only be used to train one model, not many models.

  In practice it may be difficult to estimate the error rate that can be achieved with the specific training and validation datasets. If the target error rate is too high, the training will stop before the model reaches the best state. If it is too low and unachievable, the model will train with the maximum number of iterations, which may result in overtraining. Therefore it is recommended to use the `COErrorRateMin` class to determine the lowest achievable error rate and then retrain the model with the best number of iterations.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class | Description
    --- | --- | ---
    `static class` | `COErrorRate.IterationResult` | Stores statistics for a full or partial training iteration

  - ### Field Summary

    Fields
    Modifier and Type | Field | Description
    --- | --- | ---
    `protected COErrorRate.IterationResult` | `currRes` |
    `protected COErrorRate.IterationResult` | `prevRes` |

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `COErrorRate()` | Creates object with default target error rate and max iterations.
    `COErrorRate​(double targetErrorRate, int maxIterations)` | Creates object with specified parameters.

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `void` | `beginIteration()` | Remembers result of previous iteration, prepares to accumulate results for the current iteration.
    `void` | `endIteration()` | If printing is set up, prints results of this iteration.
    `void` | `evaluatePrediction​(VectorExample e)` | Updates statistics for the current training iteration.
    `static double` | `getDftTargetErrorRate()` |
    `int` | `getMinIterations()` |
    `double` | `getProgressEstimate()` | Returns an estimate of training progress, which can be used to indicate how close the desired convergence is.
    `COErrorRate.IterationResult` | `getResult()` |
    `double` | `getTargetErrorRate()` |
    `boolean` | `isConverged()` | Returns true if the target error rate has been reached in the last training iteration, after performing at least the minimum number of iterations.
    `boolean` | `isPerfectResult()` | Returns true if all examples in the dataset are predicted correctly.
    `boolean` | `needStopTraining()` | Returns true if the target error rate (after performing the minimum number of iterations) or the maximum number of iterations have been reached.
    `void` | `printHeaderLine()` | Prints the header of the iteration list.
    `void` | `setMaxIterations​(int maxIter)` | Sets the maximum number of model training iterations to perform.
    `void` | `setMinIterations​(int minIter)` | Sets the minimum number of iterations to perform.
    `void` | `setPrintOptions​(int printAfterNIter, Partition partition, boolean printHeader)` | Enables printing of the iteration results, sets console printing options, prints header.
    `java.lang.String` | `toString()` |

    - ### Methods inherited from class com.tibco.patterns.learn.training.COIterationCount

      `getDftMaxIterations, getMaxIterations, getNIterations, hasIterations, verifyIterationBegun, verifyIterationEnded`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### currRes

          protected COErrorRate.IterationResult currRes

    - #### prevRes

          protected COErrorRate.IterationResult prevRes

  - ### Constructor Detail

    - #### COErrorRate

          public COErrorRate​(double targetErrorRate,
                             int maxIterations)

      Creates object with specified parameters.
      Parameters:
        `targetErrorRate` - - the error rate that is sufficient to stop model training.
        `maxIterations` - - maximum number of model training iterations to perform.
      Throws:
        `java.lang.IllegalArgumentException` - if maxIterations is not positive, or target error rate is not between 0 and 1.

    - #### COErrorRate

          public COErrorRate()

      Creates object with default target error rate and max iterations. Can be used when the object is treated as TrainingObserver only, i.e. we do not care about the stopping criteria.
      See Also:
        `getDftTargetErrorRate()`, `COIterationCount.getDftMaxIterations()`

  - ### Method Detail

    - #### getDftTargetErrorRate

          public static double getDftTargetErrorRate()

      Returns:
        default target error rate (0.005).

    - #### toString

          public java.lang.String toString()

      Overrides:
        `toString` in class `COIterationCount`

    - #### setMinIterations

          public void setMinIterations​(int minIter)

      Sets the minimum number of iterations to perform. It is 0 by default (not used). Can be used to extend training until the learning rate becomes reasonably small. This can avoid "lucky" results in early iterations when validation error rate happens to be the lowest, but the training error rate is still very high. Note that if zero errors in the training dataset are reached sooner, training will still stop without performing this minimum number of iterations.
      Parameters:
        `minIter` - - the minimum number of iterations to set.
      Throws:
        `java.lang.IllegalArgumentException` - if the given value is not below max iterations.

    - #### getMinIterations

          public int getMinIterations()

      Returns:
        the minimum number of iterations to perform.

    - #### setMaxIterations

          public void setMaxIterations​(int maxIter)

      Sets the maximum number of model training iterations to perform.
      Overrides:
        `setMaxIterations` in class `COIterationCount`
      Throws:
        `java.lang.IllegalArgumentException` - if the current min iterations is not below maxIter, or if maxIter is not positive.

    - #### getTargetErrorRate

          public double getTargetErrorRate()

      Returns:
        the target error rate that is sufficient to stop model training.

    - #### getResult

          public final COErrorRate.IterationResult getResult()

      Returns:
        a copy of the result of the last iteration, or null if no training was performed.

    - #### beginIteration

          public void beginIteration()

      Remembers result of previous iteration, prepares to accumulate results for the current iteration. Overriding methods must call this method.
      Specified by:
        `beginIteration` in interface `TrainingObserver`
      Overrides:
        `beginIteration` in class `COIterationCount`
      Throws:
        `java.lang.IllegalStateException` - if an iteration has already begun.

    - #### endIteration

          public void endIteration()

      If printing is set up, prints results of this iteration.
      Specified by:
        `endIteration` in interface `TrainingObserver`
      Overrides:
        `endIteration` in class `COIterationCount`
      Throws:
        `java.lang.IllegalStateException` - if iteration has not begun.

    - #### evaluatePrediction

          public void evaluatePrediction​(VectorExample e)

      Updates statistics for the current training iteration. If printing is set up, prints incorrecly classified examples after the previously specified number of iterations. Overriding methods must call this method.
      Specified by:
        `evaluatePrediction` in interface `TrainingObserver`
      Overrides:
        `evaluatePrediction` in class `COIterationCount`
      Parameters:
        `e` - - an example that must contain RLink prediction
      Throws:
        `java.lang.IllegalStateException` - if iteration has not begun.

    - #### getProgressEstimate

          public double getProgressEstimate()

      Description copied from interface: `ConvergenceObserver`
      Returns an estimate of training progress, which can be used to indicate how close the desired convergence is. The interpretation and range of the returned value can vary between subclasses. Typically, the return value approaches 0 as the model is converging. Should be called after an iteration, not in the middle of iteration.
      Specified by:
        `getProgressEstimate` in interface `ConvergenceObserver`
      Overrides:
        `getProgressEstimate` in class `COIterationCount`
      Returns:
        the error rate of the last training iteration
      Throws:
        `java.lang.IllegalStateException` - if iteration has not ended or no iterations were performed.

    - #### isConverged

          public boolean isConverged()

      Returns true if the target error rate has been reached in the last training iteration, after performing at least the minimum number of iterations.

      The error rate includes the errors for examples predicted with 0 confidence (see `COErrorRate.IterationResult.getNUntrainedPred()`). Some of these errors may be from submodels that are never trained, so model training will have no impact for these errors. Thus it is essential to eliminate untrained predictions (by adding more training examples to relevant subsets) so that the model is more likely to reach the target error rate.

      Specified by:
        `isConverged` in interface `ConvergenceObserver`
      Overrides:
        `isConverged` in class `COIterationCount`
      Returns:
        true if the error rate of the last training iteration is below the target error rate, and at least the minimum number of iterations have been performed.
      Throws:
        `java.lang.IllegalStateException` - if iteration has not ended or no iterations were performed.

    - #### needStopTraining

          public boolean needStopTraining()

      Returns true if the target error rate (after performing the minimum number of iterations) or the maximum number of iterations have been reached. See `isConverged()`.
      Specified by:
        `needStopTraining` in interface `ConvergenceObserver`
      Overrides:
        `needStopTraining` in class `COIterationCount`
      Returns:
        true if the error rate of the last training iteration is below the target error rate and at least the minimum number of iterations have been performed, or if maximum number of iterations was reached.
      Throws:
        `java.lang.IllegalStateException` - if iteration has not ended or no iterations were performed.

    - #### isPerfectResult

          public boolean isPerfectResult()

      Returns true if all examples in the dataset are predicted correctly. This is used to stop the training process based on the training dataset results: if all training examples were predicted correctly, the model cannot be trained better with these examples. This stopping criterion does not use the minimum number of iterations.
      Specified by:
        `isPerfectResult` in interface `TrainingObserver`
      Overrides:
        `isPerfectResult` in class `COIterationCount`
      Returns:
        true if all examples in the dataset were predicted correctly.
      Throws:
        `java.lang.IllegalStateException` - if iteration has not ended or no iterations were performed.

    - #### setPrintOptions

          public void setPrintOptions​(int printAfterNIter,
                                      Partition partition,
                                      boolean printHeader)

      Enables printing of the iteration results, sets console printing options, prints header. If printing is needed, this should be called before the first iteration.
      Parameters:
        `printAfterNIter` - - print errors starting with this iteration.
        `partition` - - the data partition that this object will be used with.
        `printHeader` - - if true, prints the header of the iteration list. The header for the training observer must be printed before the header for the validation observer.
      Throws:
        `java.lang.IllegalArgumentException` - if printAfterNIter is negative.

    - #### printHeaderLine

          public void printHeaderLine()

      Prints the header of the iteration list. The header for the training observer must be printed before the header for the validation observer.
      Throws:
        `java.lang.IllegalStateException` - if setPrintOptions was not called.