Solving Iteration Problems - An Example

Open the data file Hh.sta. Select Structural Equation Modeling from the Statistics - Advanced Linear/Nonlinear Models menu to display the Structural Equation Modeling Startup Panel. Select the Advanced tab, click the Open model button, and open the model file Hh.cmd. Note that both files are located in the Examples/Datasets/SEPATH subfolder of your STATISTICA installation folder.

Click the Set parameters button, and in the Analysis Parameters dialog, restore all defaults by clicking the Restore Defaults button. Under Data to analyze select the Correlations option button, and under Discrepancy function select the Generalized Least Squares (GLS) option button.

Click the OK (Accept parameters) button, and on the Startup Panel, click the OK (Run model) button to start the iteration.

Observe carefully what happens as the iteration "hangs up." After iteration terminates (as the number of iterations reaches 30), the Iteration Results window displays a message in the status bar message area at the bottom of the screen that iteration stopped because number of iterations exceeds maximum.

Click the OK button to display the Structural Equation Modeling Results dialog. Select the Advanced tab, and click the Iteration history button to produce a spreadsheet with the information printed during iteration.

On the first iteration, the program takes a relatively modest step (.265) resulting in a substantial drop in the discrepancy function (from 18.08 to 3.27). Unfortunately, the approximate Hessian (and associated information matrix) has become singular at the new parameter values, resulting in an NRP value of 2. The program continues iteration, but immediately iterates into an improper region, and has to constrain one or more parameters on the boundary, the number being indicated by NAIC values other than zero.

The program takes a large step on the 4th iteration, and immediately jumps into a region (again) where the Hessian is singular. Unfortunately, the program never finds its way out of this region, and iteration stalls out, with step sizes getting smaller and smaller, but no real progress being made.

One approach to solving the problem is to use automatic starting values. From the Structural Equation Modeling Startup Panel, click the Set parameters button to display the Analysis Parameters dialog. Under Data to analyze select the Correlations option button, under Discrepancy function select the Generalized Least Squares (GLS) option button, and under Initial values, select the Automatic option button.  

Click the OK (Accept parameters) button, and then the OK (Run model) button. Notice how, from the new initial values, iteration proceeds smoothly and effectively to the minimum.

Another approach will also work in this case. From the Structural Equation Modeling Startup Panel click Set Parameters. On the Analysis Parameters dialog, select Correlations as the Data to analyze, Generalized Least Squares (GLS) as the Discrepancy function, and Default for Initial values. These are the conditions that resulted in iteration "stalling out" before.

This time, we will control the step size so that the program can only take small steps. The hope is that this will keep the iteration from jumping into a problem region during the early going, when step direction is gauged less accurately.

Under Global iteration parameters, enter a value of 0.1 in the Maximum step length box. (The default value of 10000 leaves step size, in effect, completely unconstrained.) Because the steps will be small, you may need more iterations before convergence occurs, so enter 300 in the Maximum no. of iterations box

Click OK (Accept parameters), and restart the iteration by clicking OK (Run model) on the Startup Panel.  

Unfortunately, the iteration fails to converge to a minimum, encountering difficulties similar to what occurred previously. Notice that the problems started on iteration number one, where the program stepped into a region where the Hessian was singular. Now let's try inserting some steepest descent iterations, to remove the approximate Hessian from the calculation of step direction. Go back to the Analysis Parameters dialog and set Steepest descent iterations to 10 under Global iteration parameters.  

Then click OK (Accept parameters) and restart iteration by clicking OK (Run model) in the Startup Panel.

Although the iteration runs into difficulties at several points (by now you are probably learning to recognize when these occur by reading the output parameters during iteration), convergence eventually occurs after more than 50 iterations.

This example illustrates how you must be prepared to be both analytical and artistic when you attempt to solve problems encountered during iteration. Read the Iteration History results carefully, take appropriate action, and above all, don't be afraid to experiment.