Display Post-hoc Tests Results in GLM, GRM, and ANOVA

Post-hoc comparisons tests are available on the GLM, GRM, and ANOVA More Results - Post-hoc tab.

Display

The options in the Display group box control the manner in which the results for the requested post-hoc tests will be displayed.

Significant differences

Select the Significant differences option button to show a matrix of p values (significance levels) for the respective pairs of weighted marginal means, displayed in the columns and rows of the spreadsheet.

Homogeneous groups

Select the Homogeneous groups option button to display the results in a spreadsheet to indicate the homogeneous groups of means. In the first column, the spreadsheet will show the respective row/column numbers of the cell means; the means themselves are shown in the first numeric column of the spreadsheet (see also option Observed, weighted means on the GLM, GRM, and ANOVA More Results - Means tab). Then, in each remaining column of the spreadsheet homogeneous groups of means (those that are not significantly different from each other) are indicated by X's in the respective rows. Conversely, all means that are not identified as members of the same homogeneous group are significantly different from each other at p<alpha, where alpha is the user-defined p-value entered in the edit field to the right of this option (and as shown in the title of the spreadsheet). To read this table, begin with the first column following the means. As you go down the cells in that column for all cells with X's in them, all means thus "connected" by the X's comprise a homogeneous group, i.e., they are not significantly different from each other, given the current alpha level criterion. You then move to the next column (or homogeneous group), and proceed to look down through the rows. Again, all cell means "tied together" by X's in the respective rows (of that column) form the next homogeneous group.

Non-exhaustive search for homogeneous groups. When there is a large number of means for which post-hoc tests are computed, you may see the comment Non-Exhaustive Search in the header of the results spreadsheet to denote that a "simple" algorithm was used to extract homogeneous groups of means. Specifically, in most cases it can be expected that homogeneous groups are made up of adjacent elements in the sorted list of means (i.e., pairs of means that are more similar to each other also will less likely be significantly different from each other, and show a larger post-hoc probability). However, this is not always the case. For example, when there are widely unequal N's per group, or when the comparisons involve between-group and within-subject (repeated measures) comparisons, it often happens that homogeneous groups of means are made up of non-consecutive means in the order list of means, and in fact, can consist of arbitrary patterns of means over the ordered list of means.

For reasonably small numbers of means, the program will perform an exhaustive search of all combinations (groups) of means to determine those that are not significantly different from each other (given the current post-hoc criterion), i.e., that represent homogeneous groups. However, for large numbers of means, the possible numbers of combinations of means quickly becomes very large, and in this case the program will perform a simple search whereby, starting with each mean as a "root," other means that are not significantly different (from that mean) are added in a step-wise ("tree-building like") procedure to determine homogeneous groups. This algorithm will in most cases yield a complete picture of the grouping of means within the experimental design, however, it may not identify all possible non-redundant groups of means that are not significantly different from each other; in cases that the current effect for which post-hoc comparisons are computed does not involve between-group and within-subject (repeated measures) effects, and the number of observations in each group of the design is approximately the same, both procedures will yield identical results (again, because homogeneous groups will be made up of adjacent means in the ordered list of means).

Confidence intervals. Select the Confidence intervals option button to display the results in a spreadsheet where the pairwise differences between the means are shown in consecutive rows, and the columns show the respective differences, their (post-hoc) standard errors, significance levels, and (post-hoc) confidence interval. This display format is not available for the Unequal N HSD, Newman-Keuls, Duncan's, and Dunnett tests (see GLM, GRM, and ANOVA More Results - Post-hoc tab).

Critical ranges

Select the Critical ranges option button to display the results using the critical ranges. In this display format, Statistica computes critical ranges between ordered means, given the alpha level specified to the right of this option. Thus, given the ordered marginal means in the current effect, these values denote the minimum difference between any pair of means, given the current alpha level, and spanning the respective range of the ordered means (e.g., two means, that are 2 steps apart, i.e., which in the ordered list of means would be separated by one mean between them). This display option is only available for the Newman-Keuls and Duncan tests (see GLM, GRM, and ANOVA More Results - Post-hoc tab.