Post-hoc Comparisons of Means

The breakdowns facility in the Basic Statistics and Tables module also offers all common post-hoc tests for means comparisons. Usually, after obtaining a statistically significant F-test from the ANOVA, you want to know which of the means contributed to the effect (such as which groups are particularly different from each other).

You could perform a series of simple t-tests to compare all possible pairs of means. However, such a procedure would capitalize on chance, and the reported probability levels would actually overestimate the statistical significance of mean differences.

  • Without going into too much detail, suppose you take 20 samples of 10 random numbers each, and computed 20 means.
  • Then, take the group (sample) with the highest mean and compare it with that of the lowest mean. The t-test for independent samples tests whether or not those two means are significantly different from each other, provided they were the only two samples taken. Post-hoc comparison techniques, on the other hand, specifically take into account the fact that more than two samples were taken.