Statistical Test

A test used to determine the statistical Significance of an observation. Two main types of error can occur:

1. A Type I Error occurs when a false negative result is obtained in terms of the Null Hypothesis by obtaining a false positive measurement.

2. A Type II Error occurs when a false positive result is obtained in terms of the Null Hypothesis by obtaining a false negative measurement.

The probability that a statistical test will be positive for a true statistic is sometimes called the test's Sensitivity, and the probability that a test will be negative for a negative statistic is sometimes called the Specificity. The following table summarizes the names given to the various combinations of the actual state of affairs and observed test results.

result	name
true positive result	Sensitivity
false negative result	1-Sensitivity
true negative result	Specificity
false positive result	1-Specificity

Multiple-comparison corrections to statistical tests are used when several statistical tests are being performed simultaneously. For example, let's suppose you were measuring leg length in eight different lizard species and wanted to see whether the Means of any pair were different. Now, there are $8!/2!6! = 28$ pairwise comparisons possible, so even if all of the population means are equal, it's quite likely that at least one pair of sample means would differ significantly at the 5% level. An Alpha Value of 0.05 is therefore appropriate for each individual comparison, but not for the set of all comparisons.

In order to avoid a lot of spurious positives, the Alpha Value therefore needs to be lowered to account for the number of comparisons being performed. This is a correction for multiple comparisons. There are many different ways to do this. The simplest, and the most conservative, is the Bonferroni Correction. In practice, more people are more willing to accept false positives (false rejection of Null Hypothesis) than false negatives (false acceptance of Null Hypothesis), so less conservative comparisons are usually used.

See also ANOVA, Bonferroni Correction, Chi-Squared Test, Fisher's Exact Test, Fisher Sign Test, Kolmogorov-Smirnov Test, Likelihood Ratio, Log Likelihood Procedure, Negative Likelihood Ratio, Paired t-Test, Parametric Test, Predictive Value, Sensitivity, Significance Test, Specificity, Type I Error, Type II Error, Wilcoxon Rank Sum Test, Wilcoxon Signed Rank Test