# Logical Extremes, Beta, and the Power of the Test

James V. Pinto, Pin Ng, and David S. Allen
Northern Arizona University

Journal of Statistics Education Volume 11, Number 1 (2003), jse.amstat.org/v11n1/pinto.html

Copyright © 2003 by James V. Pinto, Pin Ng, and David S. Allen, all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.

Key Words: Power function; Simulations; Type II error.

## Abstract

There is a potential misuse of the power function under the logical extreme when the null hypothesis is true. The power function is defined to measure the probability of rejecting the null given any value of the parameter being tested. It can be used to obtain the power and the values only under the alternative hypothesis. When the null is true, the power function can be used to obtain the size of the test. The power and the probability of committing a Type II error are, however, undefined and, hence, the power function should not be used to obtain these values.

## 1. Introduction

It is not an easy or smooth task to calculate the power of a test for beginning statistics students. Until computer algorithms and simulations were designed to help in this process (see, for example, Doane, Mathieson, and Tracy 2001), the best one could do was to calculate a single point on the power function representing 1 - (as in Kvanli, Ravur, and Guynes 2000). Yu and Behrens (1994) illustrate the usefulness of simulations in determining the power of a test. It is possible to calculate every point on the power curve, but in-class demonstrations are usually limited to single points due to computational constraints and the repetitive nature of the process. Logical extremes, as detailed below, are usually ignored. The purpose of this note is to demonstrate that the logical extremes are nontrivial and that one of the extremes results in calculations of and the power of the test that are inconsistent with the common pedagogy. A detailed history of the nature of hypothesis testing, including a section of the power of the test in light of the Fisher approach versus the Neyman-Pearson approach, can be found in Lehmann (1993). He concluded that combining the best features of both could unify the two approaches. This historical debate will not be addressed in this paper.

## 2. Calculation Process

We will use the familiar test on the population mean of a normal distribution with a known variance to set up the stage for logical extremes. We assume that the data generating process is where is the probability density function of a normal distribution with an unknown mean and a known variance .

We first consider testing the simple null hypothesis H0: = = 1.5 against a two-sided composite alternative H1:   assuming = 0.1 and n = 36. In our example, under the null hypothesis, the sampling distribution of the sample mean is normal, centered at = 1.5, with a standard error of . This is illustrated by the normal distribution on the left in Figure 1, which was generated using Visual Statistics (Doane, Mathieson, and Tracy 2001). The size of the z-test (the probability of committing a Type I error), , is chosen to be 10%. Figure 1 also presents a second normal distribution on the right, that is the sampling distribution of the sample mean at the specific value of = 1.51 when the null hypothesis is false. The shaded area under this sampling distribution in the middle acceptance region measures the probability of committing a Type II error while the shaded areas in the rejection regions provide the power of the test, 1 - .

Figure 1. Beta and the Power of the Test for Two-Tailed Test.

The power function, defined as the probability of rejecting the null given a specific value of , for the z test is given by , where is the cumulative distribution function of the standard normal distribution. This yields 1 - = 0.16 on the power function at this single point of = 1.51. The entire power function is shown in Figure 2 and resembles an upside down normal distribution. It should be noted that the power function in this example approaches one asymptotically as we move away from 1.5 in either direction.

Figure 2. The Power Function for a Two-Tailed Test.

For the one-tailed test, we consider the composite null hypothesis H0:   versus the composite alternative H1: > . The sampling distributions of the sample mean under the null of = = 1.5 and the specific alternative of = 1.51. are illustrated, respectively, by the left and right distributions in Figure 3. The power function is given by . The power of the test at = 1.51. is 0.25 and the whole power function is illustrated in Figure 4. The power function in this example approaches one and zero asymptotically as we move away from 1.5 in the positive and negative directions, respectively.

Figure 3. Beta and the Power of the Test for One-Tail Test.

Figure 4. The Power Function for a One-Tail Test.

## 3. The Logical Extremes

The first logical extreme occurs when the two sampling distributions in both Figure 1 and 3 pull apart to a point where there is very little overlap between them. In this case the power of the test approaches 1 (as the two distributions in Figure 1 and Figure 3 separate), and the approaches zero. This can easily be seen in Figure 2 when moves away from = 1.5 in both directions and also in Figure 4 when moves away from = 1.5 to the right.

The second logical extreme occurs when approaches = 1.5. One subtle point that has not usually been made explicitly in the definition of the power function is the fact that the power function yields the power of a test only when the null hypothesis is false. Under the null hypothesis, the power function provides the probability of committing a Type I error instead of the power of the test. As presented in Figure 2, the power function yields the 10% size at = = 1.5 according to the accepted pedagogy (Freund and Walpole 1987).

This is where this logical extreme breaks down. In this case, it is incorrect to say that “the power of the test is 10% or the probability of committing a Type II error is 90%” since the power is defined as “the probability of rejecting a false null hypothesis” and a Type II error is defined to be “the failure to reject the false null hypothesis.” But there is no false null hypothesis when the null hypothesis is true at = = 1.5. should be undefined rather than having a maximum value of 90%. Thus, the contention that the power of the test is at a minimum of 10% under the null cannot be supported because there is no false null hypothesis when the null hypothesis is true at = = 1.5. Both the power and the probability of committing a Type II error are undefined under this logical extreme. Unfortunately, this is usually overlooked. For example Figure 5 is generated for = = 1.5 using Visual Statistics which labels the area under the “accept” region as “Type II” error and the area under the “Reject” regions as “Power.” This misconception is also apparent in the few current textbooks that attempt to address the power function. In Berenson, Levine and Krehbiel (2002, pp. CD9-6) they state “Interestingly, if the population mean is actually 368 grams, the power of the test is equal to , the level of significance (which is 0.05 in this example), because the null hypothesis is actually true.” In Newbold, Carlson and Thorne (2003, p. 357), the authors make a similar statement that includes “The power of the test at equals , ...”

Figure 5. Logical Extreme: Beta and the Power of the Test for Two-Tailed Test.

Similarly, the sampling distribution and the power function for the one-tailed test are presented in Figures 6 and 7.

Figure 6. Logical Extreme: Beta and the Power of the Test for One-Tail Test (Right).

Figure 7. Logical Extreme: The Power Function for a One-Tail Test (Right).

The power function in Figure 7 provides the probability of rejecting the null hypothesis over the whole range of values of . It can be used to obtain the power of the test only when > 1.5. The size of the test is defined as the supremum of the power function over  1.5. This yields a size of 10% according to accepted pedagogy (Mood, Graybill and Boes 1974; Tamhane and Dunlop 2000 ). The power and remain undefined under the null when  1.5.

## 4. Conclusions

There is a potential misuse of the power function under the logical extreme when the null hypothesis is true. The power function is defined to measure the probability of rejecting the null given any value of the parameter being tested. It can be used to obtain the power and the values only under the alternative hypothesis. When the null is true, the power function can be used to obtain the size of the test. The power and the probability of committing a Type II error are, however, undefined and, hence, the power function should not be used to obtain these values.

## Acknowledgments

The authors wish to thank two anonymous referees for their detailed and useful comments. All remaining errors are the responsibility of the authors.

Pin Ng