Workspace Node: SANN Regression/Time Series Regression - Specifications - Quick Specification Tab

In the SANN Regression or the SANN Time Series (Regression) node dialog box, under the Specifications heading, select the Quick Specification tab; the options change on this tab according to whether Automated network search (ANS), Custom neural networks (CNN), or Subsampling (random, bootstrap) is selected on the Quick tab.

Automated Network Search (ANS)

Element Name Description
Network types Use the options in this group box to specify the type of network (MLP or RBF). For each selected type, you can also specify a range for the complexity of the neural network models to be tried by the Automated Network Search (ANS). Specify the complexity of networks to be tested in terms of a range of figures for the number of hidden units. Specifying the number of hidden units exactly (i.e., by setting the minimum equal to the maximum) may be beneficial if you know, or have good cause to suspect, the optimal number. In this case, it enables the Automated Network Search (ANS) to concentrate its search algorithms on other dimensions in the search such as activation functions. The larger the number of hidden units in a neural network model the stronger the model is, i.e. the more capable the network is to model complex relationships between the inputs and the target variables.
MLP Select this check box to include multilayer perceptron networks in the network search. The multilayer perceptron is the most common form of network. It requires iterative training, which may be quite slow for a large number of hidden units and data sets, but the networks are quite compact, execute quickly once trained, and in most problems, yield better results than the other types of networks.
Min. hidden units Specify the minimum number of hidden units to be tried by the Automated Network Search (ANS) when using MLP networks.
Max. hidden units Specify the maximum number of hidden units to be tried by the Automated Network Search (ANS) when using MLP networks.
RBF Select this check box to include radial basis function networks in the network search. Radial basis function networks tend to be slower and larger than multilayer perceptron, and often have relatively inferior performance, but they train extremely quickly when the output activation functions are the identity. They are also usually less effective than multilayer perceptrons if you have a large number of input variables (they are more sensitive to the inclusion of unnecessary inputs).
Min. hidden units Specify the minimum number of hidden units to be tried by the Automated Network Search (ANS) when using RBF networks.
Max. hidden units Specify the maximum number of hidden units to be tried by the Automated Network Search (ANS) when using RBF networks.
Note: What effect does the number of hidden units have? In general, increasing the number of hidden units increases the modeling power of the neural network (it can model a more convoluted, complex underlying function), but also makes it larger, more difficult to train, slower to operate, and more prone to over-fitting (modeling noise instead of the underlying function). Decreasing the number of hidden units has the opposite effect.

If your data is from a fairly simple function or is very noisy, or if you have too few cases, a network with relatively few hidden units is preferable. If, in experimenting with different numbers of hidden units you find that larger networks have better training performance, but worse selection performance, you are probably overfitting and should revert to smaller networks.

To combat overfitting, SANN uses a test sample (which you can specify on the Sampling (CNN and ANS) tab) that can help the SANN training algorithm. This test sample is never used to train the neural network (i.e., to learn the data) but rather used to monitor performance throughout training at the end of each iteration cycle. See Overfitting for more details.

Train/Retain networks Use the options in this group box to specify how many networks should be trained and how many networks should be retained by the ANS.
Networks to train Specify how many networks the Automated Network Search (ANS) should perform. The larger the number of networks trained, the more detailed is the search carried out by the ANS. It is recommended that you set the value for this option as large as possible depending on your hardware speed and resources.
Networks to retain Specify how many of the neural networks tested by the Automated Network Search (ANS) should be retained (for testing, and then insertion into the current network set). Networks with the lowest error for regression and highest classification rate for classification will be retained.
Error function Specify the error function to be used in training a network.
Sum of squares This check box is selected by default to generate networks using the sum of squares error function. This error function is one of the most commonly used in training neural networks and available in SANN for both MLP and RBF types of networks in both regression and classification tasks.

Custom Neural Networks (CNN)

Element Name Description
Network type Use the options in this group box to specify the type of network (Multilayer perceptron or Radial basis function).
Multilayer perceptron (MLP) Select this option button to generate multilayer perceptron networks. The multilayer perceptron is the most common form of network. It requires iterative training and the networks are quite compact, execute quickly once trained, and in most problems yield better results than the other types of networks.
Radial basis function (RBF) Select this option button to generate radial basis function networks. Radial basis function networks tend to be slower and larger than multilayer perceptron and often have inferior performance, but they can be trained faster than MLP for large data sets and linear output activation functions.
Error function Specify the error function to be used in training a network.
Sum of squares This is the only error function available for regression type analyses, and is selected by default.
Activation functions Use the options in this group box to select activation functions for the hidden and output neurons. The choice of the activation function, i.e., the precise mathematical function, is crucial in building a neural network model since it is directly related to the performance of the model. Generally, it is recommended that you choose the tanh and identity functions for the hidden and output neurons for multilayer perceptron networks (default settings) when the Sum of squares error function is used. For radial basis function networks, the Hidden units are automatically set to Gaussian, and the Output units are set to Identity.
Hidden units Use this drop-down list to select the activation function for the hidden layer neurons. For multilayer perceptron networks, these include the Identity function, hyperbolic Tanh (recommended), Logistic sigmoid, Exponential, and Sine activation functions. For radial basis functions networks, a Gaussian activation function is always used for hidden neurons.
Identity With this function, the activation level is passed on directly as the output.
Tanh.Recommended. The hyperbolic tangent function (tanh) is a symmetric S-shaped (sigmoid) function, whose output lies in the range (-1, +1). Often performs better than the logistic sigmoid function because of its symmetry.
Logistic This is an S-shaped (sigmoid) curve, with output in the range (0, 1).
Exponential Uses the exponential activation function.
Sine Uses the standard sine activation function.
Gaussian Uses a Gaussian (or Normal) distribution. This is automatically selected for RBF neural networks.
Output units Use this drop-down list to select the activation functions for the hidden-output neurons. For multilayer perceptron networks, these include the Identity function (recommended), hyperbolic Tanh, Logistic sigmoid, Exponential, and Sine activation functions. For Radial basis function (RBF) networks, the choice of Output units is dependent on the selected Error function. For RBF networks with Sum of squares error function, an Identity activation function is used.
Identity.Recommended. With this function, the activation level is passed on directly as the output.
Tanh The hyperbolic tangent function (tanh) is a symmetric S-shaped (sigmoid) function, whose output lies in the range (-1, +1). Often performs better than the logistic sigmoid function because of its symmetry.
Logistic This is an S-shaped (sigmoid) curve, with output in the range (0, 1).
Exp Uses the negative exponential activation function.
Sine Uses the standard sine activation function.
Networks to train Use this option to specify how many networks the Custom Neural Network (CNN) should train. The larger the number of networks trained, the more detailed is the search carried out by the CNN. It is recommended that you set the value for this option as large as possible depending on your hardware speed and resources. Although you can create one type of network type at a time, by training more than one network you can find multiple solutions provided by the same network. Furthermore, on the Results - Predictions tab, you can combine the predictions of these networks to create ensembles. Using predictions drawn from an ensemble of networks can generally yield better results compared to the predictions of the individual networks (see Bishop 1995).
No. of neurons Specify the number of neurons in the hidden layer of the network. The more neurons the hidden layer contains, the more complex (flexible) it becomes.

Subsampling (Random, Bootstrap)

Element Name Description
Network type Use the options in this group box to specify the type of network (Multilayer perceptron or Radial basis function).
Multilayer perceptron (MLP) Select this option button to generate multilayer perceptron networks. The multilayer perceptron is the most common form of network. It requires iterative training and the networks are quite compact, execute quickly once trained, and in most problems yield better results than the other types of networks.
Radial basis function (RBF) Select this option button to generate radial basis function networks. Radial basis function networks tend to be slower and larger than multilayer perceptron and often have inferior performance, but they can be trained faster than MLP for large data sets and linear output activation functions.
Error function Specify the error function to be used in training a network.
Sum of squares This is the only error function available for regression type analyses, and is selected by default.
Activation functions Use the options in this group box to select activation functions for the hidden and output neurons. The choice of the activation function, i.e., the precise mathematical function, is crucial in building a neural network model since it is directly related to the performance of the model. Generally, it is recommended that you choose the tanh and identity functions for the hidden and output neurons for multilayer perceptron networks (default settings) when the Sum of squares error function is used. For radial basis function networks, the Hidden units are automatically set to Gaussian, and the Output units are set to Identity.
Hidden units Use this drop-down list to select the activation function for the hidden layer neurons. For multilayer perceptron networks, these include the Identity function, hyperbolic Tanh (recommended), Logistic sigmoid, Exponential, and Sine activation functions. For radial basis functions networks, a Gaussian activation function is always used for hidden neurons.
Identity With this function, the activation level is passed on directly as the output.
Tanh. Recommended The hyperbolic tangent function (tanh) is a symmetric S-shaped (sigmoid) function, whose output lies in the range (-1, +1). Often performs better than the logistic sigmoid function because of its symmetry.
Logistic This is an S-shaped (sigmoid) curve, with output in the range (0, 1).
Exponential Uses the exponential activation function.
Sine Uses the standard sine activation function.
Gaussian Uses a Gaussian (or Normal) distribution. This is automatically selected for RBF neural networks.
Output units Use this drop-down list to select the activation functions for the hidden-output neurons. For multilayer perceptron networks, these include the Identity function (recommended), hyperbolic Tanh, Logistic sigmoid, Exponential, and Sine activation functions. For Radial basis function (RBF) networks, the choice of Output units is dependent on the selected Error function. For RBF networks with Sum of squares error function, an Identity activation function is used.
Identity. Recommended. With this function, the activation level is passed on directly as the output.
Tanh The hyperbolic tangent function (tanh) is a symmetric S-shaped (sigmoid) function, whose output lies in the range (-1, +1). Often performs better than the logistic sigmoid function because of its symmetry.
Logistic This is an S-shaped (sigmoid) curve, with output in the range (0, 1).
Exp Uses the negative exponential activation function.
Sine Uses the standard sine activation function.
No. of neurons Specify the number of neurons in the hidden layer of the network. The more neurons the hidden layer contains, the more complex (flexible) it becomes.

Options / C / W. See Common Options.

OK Click the OK button to accept all the specifications made in the dialog box and to close it. The analysis results will be placed in the Reporting Documents node after running (updating) the project.