Multiplicities

A study hypothesis can usually not be evaluated by a statistical test of a single null hypothesis. The study may be based on comparisons of more than two groups and of more than one endpoint. Several, perhaps hundreds of statistical tests can then be found in a manuscript, and when multiple null hypotheses are tested, the false positive risk increases with the number of tested hypotheses. In confirmatory studies the significance level may need to be corrected for the multiplicity. One often used method is named after an Italian statistician, Bonferroni.

One common misunderstanding is that all multiplicity problems are solved by correcting the significance level for the number of group comparisons. This leads, however, to an insufficient correction when multiple endpoints are ignored.