SANN - Custom Neural Network/Subsampling - Weight Decay Tab

You can select the Weight Decay tab of the SANN - Custom Neural Network dialog box or the SANN - Subsampling dialog box to access the options described here. For information on the options that are common to all tabs (located at the top and on the lower-right side of the dialog box), see SANN - Custom Neural Network or SANN - Subsampling.

Option	Description
Weight decay	Use the options in this group box to specify the use of weight decay regularization for the input-hidden layer (MLP networks only), the hidden-output layer, or both. This option encourages the development of smaller weights, which tends to reduce the problem of over-fitting, thereby potentially improving generalization performance of the network. Weight decay works by modifying the network's error function to penalize large weights - the result is an error function that compromises between performance and weight size. Consequently, too large a weight decay term may damage network performance unacceptably, and experimentation is generally needed to determine an appropriate weight decay factor for a particular problem domain. Note: When the Radial basis functions (RBF) option button is selected on the Quick (MLP/RBF) tab, the Use hidden weight decay check box and Decay value field is unavailable.
Use hidden weight decay	Select this check box to apply weight decay regularization to the input-hidden layer weights.
Decay value	Specifies the value for the hidden layer weights. The larger the decay value the weaker the network.
Use output weight decay	Select this check box to apply weight decay regularization to the hidden-output layer weights.
Decay value	Specifies the weight decay value for the output layer weights. The larger the decay value the weaker the network.