SANN - Automated Network Search (ANS) - Weight Decay Tab

You can select the Weight decay tab of the SANN Automated Network Search (ANS) dialog box to specify the use of weight decay regularization. For information on the options that are common to all tabs, see SANN Automated Network Search (ANS).

This option encourages the development of smaller weights, which tends to reduce the problem of over-fitting, thereby potentially improving generalization performance of the network. Weight decay works by modifying the network's error function to penalize large weights - the result is an error function that compromises between performance and weight size. Consequently, too large a weight decay term may damage network performance unacceptably, and experimentation is generally needed to determine an appropriate weight decay factor for a particular problem domain.