Example 3: A 2-Level Between-Group x 4-Level Within-Subject Repeated Measures Design

This example demonstrates how to set up a repeated measures design. The use of the post-hoc testing facilities will be demonstrated, and a graphical summary of the results will be produced. Moreover, the univariate and multivariate tests will be computed.

Research Problem

The resulting design is a 2 (Factor A: Method of calibration; with 2 levels) by 4 (Factor B: Four different shapes of dials) analysis of variance. The last factor is a within-subject or repeated measures factor because it represents repeated measurements on the same subjects; the first factor is a between-groups factor because subjects will be randomly assigned to work under one or the other Method condition.

Following are instructions to do this from the ribbon bar and from the classic menus.

Ribbon bar. Select the Home tab. In the File group, click the Open arrow and from the menu, select Open Examples to display the Open a STATISTICA Data File dialog box. Double-click the Datasets folder, and then open the Accuracy.sta data set.

Next, select the Statistics tab. In the Base group, click ANOVA to display the General ANOVA/MANOVA Startup Panel.

Classic menus. Open the data file by selecting Open Examples from the File menu to display the Open a STATISTICA Data File dialog. The data file is located in the Datasets folder.

Then, from the Statistics menu, select ANOVA to display the General ANOVA/MANOVA Startup Panel.

In the Startup Panel, select Repeated measures ANOVA as the Type of analysis and Quick specs dialog as the Specification method, and then click the OK button (see the Introductory Overview for different methods of specifying designs). In the ANOVA/MANOVA Repeated Measures dialog, click the Variables button to display the standard variable selection dialog. In the Dependent variable list select B1 through B4; in the Categorical predictors (factors) list, select A. Click the OK button. Next, click the Factor codes button to display the Select codes for indep. vars (factors) dialog. Click the All button to select the codes (1 and 2) for the independent variable, and click the OK button to close this dialog and return to the ANOVA/MANOVA Repeated Measures dialog.

Now click the OK button to close this dialog, and click the OK button in the ANOVA/MANOVA Repeated Measures dialog to run the analysis and display the ANOVA Results dialog.

Let's first look at the summary table of All effects/Graphs (click the All effects/Graphs button on the Quick tab).

Select Effect B*A as shown above (even though this effect is not statistically significant), and click the OK button; also click OK in response to the prompt about the assignment of factors to aspects of the graph (i.e., accept the defaults in the Arrangement of Factors dialog).

It is apparent that the pattern of means across the levels of the repeated measures factor B is approximately the same in the two conditions A1 and A2. However, it appears that there is a particularly strong difference between the two methods for dial B4, where the confidence bars for the means do not overlap.

We are interested in comparing method A1 with method A2, for dial B4 only. So, in the Specify Contrasts for this Factor dialog, for specifying the contrast for factor A, set the contrast coefficients as shown below:

Click the OK button and on the larger Specify Contrasts for this Factor dialog, set all coefficients to 0 (to ignore the respective means in the comparison), except for B4.

Refer to General ANOVA/MANOVA and GLM Notes - Specifying Univariate and Multivariate Between-Groups Designs for additional details on the logic of testing planned comparisons.

Now click the OK button, and click the Compute button on the Comps tab. Here are the results.

It appears that, as was evident in the plot of means earlier, the two means are significantly different from each other.

To compute post-hoc tests, click the More results button to display the larger and more comprehensive Results dialog. Click on the Post-hoc tab, select effect B*A (i.e., the B by A interaction effect) in the Effect box, select the Significant differences option button as the display format in the Display group, and then click the Bonferroni button.

As you can see, using this more conservative method for testing the statistical significance of differences between means, the two B4 dials, across the levels of A are not reliably different from each other.

The post-hoc tests are further explained in the Post-hoc tests in GLM, GRM, and ANOVA topic; note also that when testing means in an interaction effect of between-group and within-subject (repeated measures) effects, there are several ways (options in STATISTICA) to estimate the proper error term for the comparison. These issues are discussed in Error Term for Post-hoc Tests in GLM, GRM, and ANOVA; see also Winer, Brown, and Michel (1991, p. 529-531) for a discussion of the Pooled MS reported in this results spreadsheet.

In this table, the means are sorted from smallest to largest, and the means that are not significantly different from each other have four "stars" (*) in the same column (i.e., they form a homogenous group of means); all means that do not share stars in the same column are significantly different from each other. Thus, and as discussed in Winer, Brown, and Michels (1991, p. 528), the only means that are significantly different from each other are the means for B2 vs. B4 and B3, and B1 from B4 and B3.

In short, univariate repeated measures ANOVA assumes that the changes across levels are uncorrelated across subjects. This assumption is highly suspect in most cases. In the present example, it is quite conceivable that subjects who improved a lot from time (dial) 1 to time (dial) 2 reached a ceiling in their accuracy, and improved less from time (dial) 2 to time (dial) 3 or 4. Given the suspicion that the sphericity assumption for univariate ANOVA has been violated, look at the multivariate statistics.

On the Summary tab, select all types of Multivariate tests (select all of the check boxes under Multivariate tests), and then click the Multiv. tests button under Within effects.

In this case, the same effect (B) is still statistically significant. Note that you can also apply the Greenhouse-Geisser and Huynh-Feldt corrections in this case without changing this pattern of results (see also Summary Results for Within Effects in GLM and ANOVA for a discussion of these tests).