Workspace Node: SANN Regression/Classification/Time Series - Specifications - MLP Tab
In the SANN Regression, SANN Classification, SANN Time Series (Regression), or SANN Time Series (Classification) node dialog box, under the Specifications heading, select the MLP tab to access the following options. This tab is available only if Custom neural networks (CNN) or Subsampling (random, bootstrap) is selected on the Quick tab, and when the Multilayer perceptron (MLP) option button is selected on the Quick Specification tab.
| Element Name | Description |
|---|---|
| Training Algorithm | Use the options in this group box to select a training algorithm and specify certain options that are related to the selected algorithm. The Learning rate and Momentum options are only enabled when the algorithm is Gradient descent. |
| Algorithm | From this drop-down list, you can select the training algorithm to use. |
| Gradient descent | Gradient descent is a first order optimization algorithm that attempts to move incrementally to successively lower points in search space in order to locate a minimum. |
| BFGS | Broyden-Fletcher-Goldfarb-Shanno (BFGS), or Quasi-Newton is a powerful second order training algorithm with very fast convergence but high memory requirements due to storing the Hessian matrix. |
| Conjugate gradient | Conjugate gradient is a fast training algorithm for multilayer perceptrons that proceeds by a series of line searches through error space. Succeeding search directions are selected to be conjugate (non-interfering). It is a good generic algorithm with generally fast convergence. |
| Cycles | In the Cycles field, specify the number of training cycles for the network. In each training cycle the entire training set is passed through the networks and the network error is calculated. This information is then used to adjust the weights so that the error is further reduced. |
| Learning rate | In this field, specify the learning rate used to adjust the weights. A higher learning rate may converge more quickly, but may also exhibit greater instability. Values of 0.1 or lower are reasonably conservative. Higher learning rate may cause divergence of the weights. You can only specify a learning rate when the Gradient descent algorithm has been selected. |
| Momentum | In this field, specify the momentum, which is used to compensate for slow convergence if weight adjustments are consistently in one direction - the adjustment "picks up speed." Momentum usually increases the speed of convergence of Gradient descent considerably, and a higher rate can allow you to decrease the learning rate to increase stability without sacrificing much in the way of convergence speed. You can only specify the Momentum when the Gradient descent algorithm has been selected. |
Network randomization. Use the options in this group box to specify how the weights should be initialized at the beginning of training. You can select Normal randomization or Uniform randomization. In addition to selecting a distribution, you must also specify the mean/min and variance/max to use. You can change the default mean/min and variance/max settings, but it is generally recommended that you set the mean/min to zero and variance/max no more than 0.1. This will help the network to gradually grow from its linear state (small weight values) to the nonlinear (large weight values) mode for modeling the input-target relationship as and when necessary during the training process
Normal randomization. Select the Normal randomization option button to use a normal randomization of weights for the neural network model. A normal distribution (with the mean and variance specified below) will be used to draw the initial weight values.
Uniform randomization. Select the Uniform randomization option button to use a uniform randomization of weights for the neural network model. A uniform distribution (with the mean and variance specified below) will be used to draw the initial weight values.
Mean/Min. In this field, specify either the mean (for the normal distribution) or the minimum value (for the uniform distribution) to use for drawing the initial (i.e., before training starts) weight sample.
Variance/Max. In this field, specify either the variance (for the normal distribution) or the maximum value (for the uniform distribution) to use for drawing the initial (i.e., before training starts) weight sample.
Stopping conditions. Use the options in this group box to specify when to apply the stopping conditions for early stopping of network training. Note: Although SANN defaults to using the training set for early stopping when no test sample is selected, it is still possible to use the training set for that purpose while also having a test sample. To do so, select a validation set (instead of a test set). The only difference between test and validation sets is that the former is used for early stopping while the latter is never presented to the network while training is in progress. So, by having a validation set, you effectively have a test set while using the training sample for early stopping.
Enable stopping conditions. Select this check box to implement early stopping to the training of the neural network. Early stopping is applied when the conditions defined below are met by the training algorithm.
Change in error. When stopping conditions are applied, network training will end if the average network error improvement over a specified number of training cycles (see next entry) is less than the Change in error value given here.
Window. In the Window field, enter the number of training cycles over which the average improvement in network error must be at least as large as the specified Change in error (see above).
Options / C / W. See Common Options.