Example 6: Repeated Measures ANOVA Design

Research Problem

For example, operators of complex industrial machinery (e.g., nuclear power plants) constantly need to read (and process) various gauges and adjust machines (dials) accordingly. It would be interesting to learn how people's performance on this task is affected by white noise (which sounds like the hiss you hear when tuning your radio in between stations) as compared to meaningful background noise (if you tune the radio to an actual station).

This will be simulated in an experiment in which subjects are required to adjust one of three different dials whenever the respective gauge indicates a significant deviation from specification. Some subjects will perform this task under white background noise conditions, while others will perform it under meaningful background noise (e.g., a radio news program). The dependent measure is the number of errors that the subjects make (failure to set a dial properly within a short tolerance period) during three consecutive 10-minute time intervals.

It may be convenient to order those variables in a meaningful manner, but no special order is necessary because the GLM module allows you great flexibility in specifying the levels of the repeated measures factors. In this example data file, there are 6 subjects (3 in each group), and the order of variables are arranged so that for each successive time period the measurements for each dial are placed next to each other. The next image shows the data file Dials.sta.

Classic menus. From the File menu, select Open Examples to display the Open a STATISTICA Data File dialog box. Open the data file, which is located in the Datasets folder. Then, from the Statistics - Advanced Linear/Nonlinear Models submenu, select General Linear Models to display the General Linear Models (GLM) Startup Panel.

In the Type of analysis box, select Repeated measures ANOVA. In the Specification method box, select Quick specs dialog. Then, click the OK button to display the GLM Repeated Measures ANOVA Quick Specs dialog box.

Click the Variables button to display a standard variable selection dialog box.

Select variables 2 - 10 (Tim1_Dl1 to Tim3_Dl3) in the Dependent variable list (repeated measures variables). Select variable 1 (Noise) as the Categorical predictor (factors), and then click the OK button.

Next, click the Factor codes button and select the codes (MEANGFLL and WHITE) for the categorical predictor variable by clicking the All button and then the OK button.

In this example, there are two repeated measures factors: 1) the three successive 10-minute time periods, and 2) the three dials. To specify these factors, click the Within effects button, and in the resulting Specify within-subjects factors dialog box, specify the repeated measures factor names and their levels, as described below.

When you look at the data file, the slowest changing subscript in the list of variables is that specifying the levels of the Time factor. To clarify the term "subscript," look back at the spreadsheet of the data file, and while moving your finger over the list beginning with variable Tim1_Dl1 say out loud "one two three one two three one two three." Each "one" in this case points to a variable that contains data for the first dial (Dl1), each "two" for the second dial (Dl2), and each "three" for the third dial (Dl3). Thus, the fastest changing subscript is that of factor Dial, and you must specify that factor last. Consequently, factor Time must be specified first.

Thus, in the Specify within-subjects factors dialog box, enter 3 as the No. of levels and TIME as the first Factor Name; likewise, enter 3 as the No. of levels for the second Factor Name - DIALS.

You have now specified the within-subject (repeated measures) part of the design; click the OK button to return to the GLM Repeated Measures ANOVA Quick Specs dialog box.

To view the syntax program automatically generated from the specifications, click the Syntax editor button in the lower-right corner of the GLM Repeated Measures ANOVA Quick Specs dialog box to display the GLM Analysis Syntax Editor.

The remainder of the specifications for this analysis can use the default specifications, so click the OK (Run) button in the GLM Analysis Syntax Editor or the OK button in the GLM Repeated Measures ANOVA Quick Specs dialog box to perform the analysis.

In the ANOVA table, there are three significant effects [with an asterisk (*) by the p-value] in this analysis: the main effect for Time, the Time*Noise interaction, and the main effect for Dials. Now, look at the significant interaction.

First, select (i.e., highlight) the Time*Noise interaction, and then select the Spreadsheet option button in the Display group box.

Click the OK button to produce the spreadsheet of marginal means for this effect.

The easiest (and sometimes it seems only) way to understand interactions is to plot them. Therefore, return to the Table of All Effects (click the All effects/Graphs button), and select the Graph option button in the Display group box. Now, click the OK button to display the Arrangement of Factors dialog box.

Here you can rearrange the factors for the plot, however, for this example, just click the OK button and accept the default selections.

It appears that in both Noise conditions, subjects' performance improved, that is, they made fewer errors in successive 10-minute trials. However, the two lines denoting the two Noise conditions diverge starting at time period 2 (level_2). It seems that subjects did better (made fewer errors) in the white noise condition than in the meaningful noise condition at Times 2 and 3, but not at Time 1.

For this example, the difference between the two noise conditions, separately at Times 2 and 3, will be evaluated.

Click the Contrasts for LS means button to display the Specify Contrasts for this Factor dialog box.

First, specify a contrast for factor Noise: enter 1 and -1, respectively (in order to contrast the first level with the second level). Note that instead of typing the values of contrast coefficients, you can use the Quick Fill facility, which enables you to enter the coefficients (into individual cells or entire columns) by clicking the respective buttons.

Click the OK button to display the Repeated Measures Factors dialog box, listing each of the repeated measures factors.

In order to enter the contrast for the factor Time, click the Time button. Then enter the coefficients 0, 1, and 0, respectively, in the Enter Contrasts for this Factor dialog box. (Note that you can use the Quick Fill facility, as explained above.)

Click the OK button to return to the Repeated Measures Factors dialog box and similarly click the Dials button. Enter 1, 1, and 1 as the coefficients for this factor. In this manner, you will collapse across all levels of this factor. Because these are the default coefficients, you may also simply click the OK button in the Enter Contrasts for this Factor dialog. Now, click the OK button in the Repeated Measures dialog box to return to the Comps tab. Finally, click the Compute button to produce numerous spreadsheets. In particular, we will look at the Univariate Test of Significance for Planned Comparison spreadsheet.

Apparently, the difference between the two means (White vs. Meaningful noise) at Time 2 is not statistically significant. You may also want to try the comparison between those two groups at Time 3 (for factor Time enter the coefficients 0, 0, 1; all other specifications are the same). That comparison is not significant either.

The significance of the interaction between Noise and Time (ignoring the third level of Time, i.e., Time 3) will now be tested. To do this, once again, click the Contrasts for LS means button and specify the contrast coefficients 1 and -1 for the Noise variable in the Specify Contrasts for this Factor dialog box and click the OK button.

Next, enter the coefficients 1, -1, 0, respectively, for factor Time and the coefficients 1, 1, 1 for factor Dials in the Repeated Measures Factors dialog box (see above) and then click the OK button. As you can see, the first two levels of factor Noise and the first two levels of factor Time are being contrasted. Now, click the Compute button to view the spreadsheets.

This contrast is statistically significant. It indeed looks as though the significant two-way interaction between Noise and Time can be traced to the differential changes (improvements) in the number of errors from Time 1 to Time 2 in the White noise condition as compared to the Meaningful condition; subjects in the former condition improved their performance more than subjects in the latter condition.

In short, univariate repeated measures ANOVA assumes that the changes across levels are uncorrelated across subjects. This assumption is highly suspect in most cases. In the present example, it is quite conceivable that subjects who improved a lot from Time 1 to Time 2 reached a ceiling in their accuracy, and improved less from Time 2 to Time 3. Given the suspicion that the sphericity assumption for univariate ANOVA has been violated, look at the multivariate statistics.

There are different multivariate test criteria; in this case, Wilks' Lambda, Pillai-Bartlett Trace, Hotelling-Lawley Trace, and Roy's Largest Root agree: the interaction is not significant at the .05 level! Thus, here you have a case where the violation of the sphericity assumption led to an erroneous acceptance of the interaction effect as being statistically significant.

The spreadsheet shows the adjusted univariate tests. As you can see above, the Greenhouse-Geisser adjustment does a "good job" of protecting against erroneously accepting the interaction as statistically significant at the .05 level. In fact, the p-value for that test (.057) is very similar to that for the multivariate test (p = .062). The Huynh-Feldt adjustment does not help much in this case.

See also GLM - Index.