Example 1: Standard Regression Analysis

The information for each variable is listed in the Variable Specifications Editor (accessible by selecting All Variable Specs from the Data menu).

Now, click the OK button in this dialog and the Review Descriptive Statistics dialog box is displayed. Here, you can review the means and standard deviations, correlations, and covariances between variables. Note that this dialog is also available from basically all subsequent Multiple Regression dialog boxes, so you can always come back to look at the descriptive statistics for specific variables. Also, there are numerous graphs available.

Select Histograms from the Graphs menu to produce the following histogram of variable PT_POOR. On the 2D Histograms dialog box - Advanced tab, in the Intervals group box, select the Categories option button, enter 16 in the corresponding edit field, and then click the OK button. In the variable selection dialog, select PT_POOR, and click OK. As you can see in the following image, the distribution for this variable deviates somewhat from the normal distribution. Correlation coefficients can become substantially inflated or deflated if extreme outliers are present in the data. However, even though two counties (the two right-most columns) have a higher percentage of families below the poverty level than what would be expected according to the normal distribution, they still seem to be sufficiently "within range."

This decision is somewhat subjective; a rule of thumb is that one needs to be concerned if an observation (or observations) falls outside the mean ± 3 times the standard deviation. In that case, it is wise to repeat critical analyses with and without the outlier(s) to ensure that they did not seriously affect the pattern of intercorrelations. To view the distribution of this variable, click the Box & whisker plot button on the Review Descriptive Statistics dialog box - Advanced tab. In the variable selection dialog, select the variable Pt_Poor and click OK. Select the Median/Quart./Range option button in the Box-Whisker Type dialog box, and then click the OK button to produce the box and whisker plot.   

(Note that the specific method of computation for the median and the quartiles can be configured "system-wide" on the General tab of the Options dialog box.)

Return to the Review Descriptive Statistics dialog box and click the Correlations button on the Quick tab or the Advanced tab to display the spreadsheet with the correlation matrix.

The correlations between variables can also be displayed in a matrix scatterplot. A matrix scatterplot of selected variables can be produced by clicking the Matrix plot of correlations button on the Review Descriptive Statistics dialog box - Advanced tab and then selecting the desired variables.

This spreadsheet shows the standardized regression coefficients (b*) and the raw regression coefficients (b). The magnitude of these Beta coefficients enable you to compare the relative contribution of each independent variable in the prediction of the dependent variable. As is evident in the spreadsheet shown above, variables POP_CHNG, PT_RURAL, and N_EMPLD are the most important predictors of poverty; of those, only the first two variables are statistically significant. The regression coefficient for POP_CHNG is negative; the less the population increased, the greater the number of families who lived below the poverty level in the respective county. The regression weight for PT_RURAL is positive; the greater the percent of rural population, the greater the poverty level.

The semi-partial correlation is the correlation of the respective independent variable adjusted by all other variables, with the raw (unadjusted) dependent variable. Thus, the semi-partial correlation is the correlation of the residuals for the respective independent variable after adjusting for all other variables, and the unadjusted raw scores for the dependent variable. Put another way, the squared semi-partial correlation is an indicator of the percent of Total variance uniquely accounted for by the respective independent variable, while the squared partial correlation is an indicator of the percent of residual variance accounted for after adjusting the dependent variable for all other independent variables.

In this example, the partial and semi-partial correlations are relatively similar. However, sometimes their magnitude can differ greatly (the semi-partial correlation is always lower). If the semi-partial correlation is very small, but the partial correlation is relatively large, then the respective variable may predict a unique "chunk" of variability in the dependent variable (that is not accounted for in the other variables). However, in terms of practical significance, this chunk may be tiny and represent only a very small proportion of the total variability (see, for example, Lindeman, Merenda, and Gold, 1980; Morrison, 1967; Neter, Wasserman, and Kutner, 1985; Pedhazur, 1973; or Stevens, 1986).

The scale used in the casewise plot in the left-most column is in terms of sigma, that is, the standard deviation of residuals. If one or several cases fall outside of the ± 3 times sigma limits, one should probably exclude the respective cases (which is easily accomplished via case selection conditions) and run the analysis over to make sure that key results were not biased by these outliers.

Statistica provides an interactive outlier removal tool (the Brushing Tool ) in order to experiment with the removal of outliers to instantly see their influence on the regression line. Right-click in the graph, and select Show Brushing from the shortcut menu. When the tool is activated, the cursor changes to a cross-hair, and the Brushing dialog box will be displayed next to the graph. You can (temporarily) interactively eliminate individual data points from the graph by selecting 1) the Auto Apply check box and 2) the Off option button in the Action box; then click on the point that is to be removed with the cross-hair. Clicking on a point will automatically remove it (temporarily) from the graph.

As previously mentioned, multiple linear regression assumes linear relationships between the variables in the equation, and a normal distribution of residuals. If these assumptions are violated, your final conclusion may not be accurate. The normal probability plot of residuals will give you an indication of whether gross violations of the assumptions have occurred. Click the Normal plot of residuals button on the Probability plots tab to produce this plot.

This plot is constructed as follows. First the residuals are rank ordered. From these ranks, z values can be computed (i.e., standard values of the normal distribution) based on the assumption that the data come from a normal distribution. These z values are plotted on the y-axis in the plot.

If the observed residuals (plotted on the x-axis) are normally distributed, then all values should fall onto a straight line in the plot; in this plot, all points follow the line very closely. If the residuals are not normally distributed, then they will deviate from the line. Outliers may also become evident in this plot.

If there is a general lack of fit, and the data seem to form a clear pattern (e.g., an S shape) around the line, then the dependent variable may have to be transformed in some way (e.g., a log transformation to "pull in" the tail of the distribution, etc.; see also the brief discussion of Box-Cox and Box-Tidwell transformations in the Regression Notes). A discussion of such techniques is beyond the scope of this example (Neter, Wasserman, and Kutner, 1985, page 134, present an excellent discussion of transformations as remedies for non-normality and non-linearity); however, too often researchers simply accept their data at face value without ever checking for the appropriateness of their assumptions, leading to erroneous conclusions. For that reason, one design goal of the Multiple Regression module was to make residual (graphical) analysis as easy and accessible as possible.