Example 4: Estimating the Population Intraclass Correlation via Variance Components

This example illustrates how to estimate the population intraclass correlation coefficient for a random factor using variance component estimation (Hays, 1988, pp. 483-490). The value of the population intraclass correlation coefficient is a measure of the homogeneity of observations within the classes of a random factor relative to the variability of such observations between classes. It will be zero when the estimated effect of the random factor is zero and will reach unity only when the estimated effect of error is zero, given that the total variation of the observations is greater than zero (see Hays, 1988, p. 485).

Example 4 is based on an example data set presented by Hays (1988, p. 484). Five different experimenters chosen at random each administered 2 tests to 8 subjects. The independent variable is Tester and the dependent variables are Y1 and Y2. Y1 is the same dependent variable presented in the example by Hays (1988, p. 484). There are significant differences in the means on Y1 for different Testers (see Table 13.4.2 in Hays, 1988, p. 484). Y2 is a transformation of Y1 such that it has the same within-Tester variation as Y1, but identical means for each Tester. The data are available in the example data file hays484.sta (a partial listing of this data file is shown below). Open this data file via the File - Open Examples menu; it is in the Datasets folder.

To perform the analysis, select Variance Components from the Statistics - Advanced Linear/Nonlinear Models menu to display the Variance Components & Mixed Model ANOVA Startup Panel. On the Quick tab, click the Variables button to display the standard variable selection dialog. Here, select variable Y1 and Y2 as the Dependent vars, variable Tester as the Random factors, and then click the OK button. All other options will remain at their default settings; therefore, click the OK button to display the Variance Components and Mixed Model ANOVA/ANCOVA Results dialog.

Reviewing the Results.

For this example we will estimate variance components using the ANOVA-based Type I Expected mean squares method (see Variance Components and Mixed Model ANOVA/ANCOVA Method Options). On the Advanced tab, select the Expected MS option button under Method and the Type I option button under SS Type. Then click the Summary: Components of variance button to display the Components of Variance spreadsheet shown below. (Note that additional spreadsheets may also be displayed depending on the selection of the check boxes under the Summary: Components of variance button.)

The population intraclass correlation coefficient for Tester on Y1 is computed as the ratio of the estimated variance component for Tester on Y1 to the sum of the Tester and Error variance components on Y1. For this example the population intraclass correlation coefficient for Tester on Y1 is computed as .098645 / (.098645 + .081214), or .55, indicating that 55% of the variation on Y1 is accounted for by Tester.

Similarly, the calculations for the population intraclass correlation coefficient for Tester on Y2 are -.010152 / (-.010152 + .081214), or -.14. This cannot logically be taken as indicating that -14% of the variation on Y2 is accounted for by Tester, because variation, by definition, is positive. It is instead taken as indicating that the variance component for Tester is zero, that the population intraclass correlation coefficient for Tester on Y2 is zero, and that 0% of the variation on Y2 is accounted for by Tester. The seemingly negative variance component for Tester on Y2 can easily be understood by comparing the Mean squares for Tester and for Error from the ANOVA. Y2 has exactly the same within Tester class variation as Y1, but no differences in Tester class means. Thus the Mean square for Tester on Y2 is zero, which is less than the Mean square for Error. The negative variance component estimate for Tester on Y2 simply reflects the fact that observations are less homogeneous (i.e., have more variation) within classes of Tester than between classes of Tester.

These relative variances can be interpreted as zero-order intraclass correlations when there is only one random factor in the analysis. If there is more than one random effect in the analysis and the random effects are correlated, the relative variances should be interpreted as partial intraclass correlations.

See Variance Components Index.