What Is the Meaning of a Critical Region in Biostatistics?

The Basic Idea of Hypothesis Testing

• The basic notion of hypothesis testing can be looked at like a game or trial between one side advocating for the null hypothesis (that is, nothing is going on) and another side advocating for the opposite---that the data is not compatible with the null hypothesis---that something is, in fact, going on. Neither side can ever have conclusive statistical evidence, but each side's evidence can be stronger or weaker.

The Critical Region and Hypothesis Testing

• Similar to a trial where you assume the accused is "innocent until proven guilty," in standard hypothesis testing you assume the null is true until proven false. The standard method of showing that it is false is to compute a test statistic (e.g., a t-test, a correlation coefficient, an F-test or any other statistical test) and see how likely such a test statistic is, given that the null is true. If this is less than a given number (usually 5 percent) you reject the null hypothesis. The values that result in rejection of the null are known as the critical region.

Misconceptions about the Critical Region

• The most common misconception is that the failure to have a value in the critical region is evidence that the null hypothesis is true. There is great confusion between "how likely are these data, given that the null is true" and "how likely is the null to be true, given these data". Hypothesis testing, which is what the significance region is used for, lets you answer the first question, not the second; however, you are usually more interested in the second question.

Alternatives to the Significance Region

• Ziliak and McCloskey (among dozens of others) raise important questions about the whole notion of hypothesis testing and significance regions. Rather than evaluating the statistical significance of a test result by seeing if it falls in the significance region, they suggest that you look at measures of effect size and the accuracy of those measures---that is, how big is the effect, and how good is our guess as to how big it is.

Types of Critical Regions

• Each type of statistical test will have an associated critical region. If you are comparing two means, then you may want the critical region associated with the t-statistic. If you are comparing more than two means, then you may want that associated with the F-statistic. You want to be sure to choose the appropriate critical region for your hypothesis.