|
|
Matthew A. Carlton
Cal Poly State University, San Luis Obispo
Journal of Statistics Education Volume 13, Number 2 (2005), jse.amstat.org/v13n2/carlton.html
Copyright © 2005 by Matthew Carlton, all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.
Key Words: Bayes’ Rule; Monty Hall Problem; Pedigree analysis; Prisoner’s Paradox.
The first example presented below came to my attention while co-writing a paper with W. D. Stansfield (Stansfield and Carlton (2003)). The problem, and the incorrect solution presented below, appeared in a series of letters in The Journal of Heredity between Stansfield and biologist H. W. Norton. The incorrect solution also appeared in the first edition of Stansfield (1969). Students with an interest in biology or biostatistics will find this example particularly interesting, since students rarely see the application of mathematical probability to the biological sciences in an introductory statistics course. The example requires minimal understanding of biology or genetics, and we present that information with the problem.
The second and third examples are more mathematical in nature and have been written about extensively. But the Prisoner’s Paradox and the Monty Hall Problem, as the two examples are known, both illustrate the hazards of not carefully designing a tree diagram or incorrectly assuming certain outcomes to be equally likely. Further, a certain parallelism exists in both the incorrect and correct solutions to the two problems. We will note that parallel in what follows.
All three examples share one common feature which distinguishes them from many of the examples we use in our introductory statistics courses: they are not easily put into the contingency table framework. We often motivate our students’ notion of probability by relating intersections and conditions to elements of a contingency table. In fact, this is arguably the best way to introduce these basic probability topics (see, for example, Rossman and Short 1995). Our three examples illustrate the need for other display tools, specifically the tree diagram and sample space Venn diagram, when contingency tables do not readily apply.
We should note that contingency tables, tree diagrams, and Venn diagrams do not exhaust all options. Other graphical representations for conditional probability and Bayes’ Rule have been suggested in the education literature, e.g. the reverse flow diagram (Chu and Chu 1992). However, such tools are far less standard - none appear in any textbook I have seen - a testament to the simplicity and pedagogical merit of our traditional displays.
Figure 1: An incomplete genetic pedigree.
Figure 1 shows a possible genetic pedigree. Squares denote male animals and circles denote female animals. The letters within the square or circle indicate the genetic composition, or genotype, of the animal for a particular characteristic under study.
For simplicity, suppose a single gene controls the color of hamsters: black (B) is dominant and brown (b) is recessive. Hence, a hamster will be black unless its genotype is bb. In the figure, the genotypes of both members of Generation I are known, as is the genotype of the male member of Generation II. We know that hamster II2 must be black-colored thanks to her father, but suppose that we don’t know her genotype exactly (as indicated by B- in the figure).
We want to answer the following probability question: If we observe that hamster III1 has a black coat (and hence at least one B gene), what is the probability she is genetically heterozygous; i.e., what is the probability her genotype is Bb? (Note: The symbols Bb and bB refer to the same genotype, one dominant gene and one recessive gene. Conventionally, we write Bb for this genotype unless we are trying to distinguish which parent provided the dominant gene, as illustrated below.)
We will present two solutions here: an incorrect but plausible-sounding solution, followed by the correct solution employing Bayes’ Rule. Then we will compare these two solutions by explicitly writing out the sample space for this problem. Direct examination of the sample space will illuminate where the incorrect solution goes awry.
The wrong solution: Knowing the genotypes of Generation I, we see that II2 has genotype BB or Bb with probability 1/2 each (this much is correct). If II2 is BB, then Generations I and II are identical, and the same rationale implies III1 has genotype BB or Bb with probability 1/2 each. On the other hand, suppose II2 is Bb. Two heterozygous parents can spawn four genotypes in their offspring: BB, Bb, bB, bb. Knowing III1 is black, and hence not bb, the conditional probability that III1 is heterozygous then becomes 2/3. We collect terms and find that the total probability of III1 being heterozygous, given she is black, equals
We will re-examine this solution later and see exactly why it is incorrect.
The right solution: Consider the following tree diagram.
Figure 2: A tree diagram for our pedigree.
The event H that III1 is heterozygous corresponds to the starred outcomes above. Hence,
The event B that III1 is black includes all possible outcomes except the last, whence
Finally, note that H implies B: a genetically heterozygous hamster is automatically black, because black is dominant. As a consequence, P(B | H) = 1. Thus, applying Bayes’ Rule,
What went wrong in the first solution? Let’s examine the sample space of this problem. The information given in Figure 1 allows six possible outcomes, with the associated probabilities given in Figure 2.
Figure 3: The sample space of outcomes for our pedigree, with associated probabilities.
In Figure 3, each ordered pair specifies the genotype of II2 and III1, respectively. These outcomes also appear in the tree diagram of Figure 2. We can explicitly list the outcomes in the aforementioned events H and B: H = {(BB,bB), (bB,Bb), (bB,bB)} and B = {(BB,BB), (bB,BB), (bB,Bb), (BB,bB), (bB,bB)}. Again, notice that H is actually a subset of B in this example. By adding up individual probabilities, we find, as before,
The first solution makes a key mistake: event B is used to rule out the outcome (bB,bb), but only the probabilities corresponding to II2 = bB are readjusted, as seen in Figure 4.
Figure 4: Incorrect application of conditional probability.
If we use the incorrect probabilities in Figure 4, we identify the double-starred outcomes as having III1 being heterozygous, and we arrive at the mistaken answer 1/6 + 1/4 + 1/6 = 7/12 mentioned previously.
The wrong solution fails to account for the fact that our (partial) knowledge of III1’s genotype affects the likelihood that II2 is heterozygous. After all, a black hamster is more likely to have a homozygous dominant (BB) parent than a heterozygous parent. In fact, we can use Bayes’ Rule and the correct sample space diagram Figure 3 to find the posterior probability that II2 is BB, given the information that her daughter is black. Let A denote the event that II2 is BB, i.e. A = {(BB,BB),(BB,bB)}. Notice that A is also a subset of B. Hence, we have
Compared with the prior probability P(A) = 1/2, we indeed see that a black hamster is more likely the child of a homozygous dominant parent than of a heterozygous parent.
Incorrect revision of probabilities also lies at the heart of the now-famous Monty Hall Problem. The problem was first posed by a reader in Marilyn vos Savant’s weekly Ask Marilyn column. We present the original question below. Many mathematicians criticized vos Savant’s correct solution and insisted upon the correctness of a wrong solution, also below. Check out vos Savant 1996 for a discussion of the problem and the controversy surrounding her correct solution.
| Suppose you’re on a game show, and you’re given the choice of three |
| doors. Behind one door is a car, behind the others, goats. You pick a door, |
| say number 1, and the host, who knows what’s behind the doors, opens |
| another door, say number 3, which has a goat. He says to you, “Do you |
| want to pick door number 2?” Is it to your advantage to switch your choice |
| of doors? |
| Craig F. Whitaker Columbia, MD |
The problem is intended to mimic a situation from the game show Let’s Make a Deal, hosted by Monty Hall (hence the problem’s title). Again, we will present here two solutions: a plausible wrong answer and the correct answer using Bayes’ Rule. Then, we will explore the mistake being made.
The (in)famous wrong solution: With one goat revealed, door number 2 hides one of two possible prizes: the other goat or the car. Since we have no knowledge of which it might be, the chance of finding the car behind door number 2 is 1/2, and there is no advantage to making the switch.
As we shall see, the erroneous solution again arises from misapplication of conditional probability. In particular, the solution above fails to realize that some information has been gained with the unveiling of one goat.
The right solution: Consider the tree diagram in Figure 5. Remember that Monty can neither open Door #1 (the contestant’s choice) nor open the door hiding the car.
Figure 5: Tree diagram for the Monty Hall Problem.
Let C denote the event that the car is behind Door #2; the a priori probability of C is P(C) = 1/3. Let D denote the event that Monty opens Door #3; according to Figure 5.
Hence, by Bayes’ Rule,
Therefore, the car is hidden behind the remaining door two-thirds of the time. In other words, the contestant can double his chance of winning the car (from his initial 1-in-3 guess) by employing the strategy of switching when Monty Hall gives him the option.
Let’s consider what goes wrong in the first solution. We can draw the sample space as in Figure 6.
Figure 6: Sample space diagram for the Monty Hall Problem.
In Figure 6, the first coordinate indicates the location of the car, while the second coordinate indicates the door revealed by Monty Hall. Under this notation, we have C = {(2,3)} and D = {(1,3),(2,3)}. Notice that C is a subset of D, whence
The mistake made in the first solution is to assume that the outcomes left over once we condition on C are equally likely:
Figure 7: Incorrect revision of the conditional probabilities.
This obviously leads to the false solution P(C | D) = (1/2)/1 = 1/2.
Mosteller (1965) provides the following classic example, the Prisoner’s Paradox. (Mosteller calls this problem the Prisoner’s Dilemma; however, game theorists use this title for an entirely different problem.) Of course, as we shall see, the apparent “paradox” really stems from incomplete delineation of the sample space. I have presented this problem to my senior-level probability course and found that not all students can “resolve” the paradox; those who could invariably diagrammed the problem before assessing any probabilities.
We present here a version of the Prisoner’s Paradox paraphrased from Mosteller (1965).
| Three prisoners, A, B, and C, have applied for parole. The parole board has decided to |
| release two of the three, and the prisoners know this but not which two. Prisoner A |
| realizes that it would be unethical to ask the warden if he, A, is to be released, but |
| thinks of asking for the name of one prisoner other than himself who is to be released. |
| He thinks that if the warden says “B will be released,” his own chances have now gone |
| down from 2/3 to 1/2, because either A and B or B and C are to be released. And so, A |
| decides not to reduce his chances by asking. Explain why A is mistaken in his calculations. |
The incorrect solution, and the source of the prisoner’s “paradox”: Three possible pairs of prisoners may be released, with each of the three pairs equally likely: (A & B), (A & C), (B & C). Knowledge of B’s release eliminates the option (A & C), and the conditional probability that A will be paroled becomes
The correct solution: Prisoner A has omitted from his considerations a second unknown besides the identity of the other released prisoner: the decision-making process of the warden. If we assume the parole board to be equally likely to release any two prisoners, we may diagram all possibilities as in Figure 8.
Figure 8. A tree diagram for The Prisoner’s Paradox.
Let A denote the event that Prisoner A will be paroled, and let R denote the event that the warden will reveal the intended release of Prisoner B. Prisoner A correctly assesses his a priori chance of parole to be P(A) = 1/3 x 1 + 1/3 x 1 = 2/3. According to Figure 8,
Hence, using the definition of conditional probability, we find
That is, the probability Prisoner A will be paroled is not affected by the revelation that Prisoner B will be paroled. Note that we could also have deduced P(R | A) from Figure 8 and applied Bayes’ Rule here, but using the definition of conditional probability was easier.
As Mosteller notes, Prisoner A does not have the correct sample space in mind. The correct sample space consists of four outcomes (see Figure 9), which are not all equally likely. Equivalently, he has only considered the first branch of the tree diagram, ignoring the (possible) decision the warden must make. Prisoner A believes the complete sample space has just three, equally likely, outcomes.
Figure 9: Correct sample space for The Prisoner’s Paradox.
Notice the similarities between Figure 5 and Figure 8, and Figure 6 and Figure 9. The incorrect solutions to both the Prisoner’s Paradox and the Monty Hall Problem lead to the belief that two remaining possibilities are equally likely (hence, probability 1/2). In both cases, careful diagramming and, likewise, careful application of probability rules uncover the reality that we began with four possible outcomes, not all of which are equally likely, and that the two which remain upon conditioning are likewise not equally likely.
Notice that the two problems (as we have stated them here) are also complimentary in the sense that both the contestant and the prisoner have a 2/3 chance to “win,” provided they calculate their probabilities of success correctly. Further, the correct solutions hinge on Monty Hall’s or the warden’s inability to reveal the status of the “player”: Monty cannot reveal what’s behind Door #1, and the warden cannot divulge whether Prisoner A will be paroled. (We will return to this point in the next section.)
Before presenting a formal solution to the Monty Hall Problem to my students, I find that it helps to give an intuitive explanation for the 1/3 - 2/3 solution. Imagine you plan to play Let’s Make a Deal and employ the “switching strategy.” As long as you initially pick a goat prize, you can’t lose: Monty Hall must reveal the location of the other goat, and you switch to the remaining door - the car. In fact, the only way you can lose is if you guessed the car’s location correctly in the first place and then switched away. Hence, whether the strategy works just depends on whether you initially picked a goat (2 chances out of 3) or the car (1 chance out of 3).
Numerous websites offer simulation evidence of the 1/3 - 2/3 solution; my personal favorite is www.stat.sc.edu/~west/javahtml/LetsMakeaDeal.html. Some of my colleagues and I have used these Java demos in our introductory probability classes; high school students should also find them beneficial.
The following variation can help to assess students’ understanding of this problem: Suppose that you have watched Let’s Make a Deal for years and know that the three doors are not equally likely to hide the car. Rather, the car is behind Door #1 50% of the time, behind Door #2 40% of the time, and behind Door #3 10% of the time. What is your best strategy? That is, which door should you pick originally, and should you stay or switch? If students really understand the problem and its solution, they will realize they should choose Door #3 (the least likely door) and then switch. By the same logic as above, this strategy loses a mere 10% of the time.
As noted before, the “trick” behind the Monty Hall Problem, i.e. what makes an intuitively pointless move advantageous, is the restriction that Monty cannot reveal the location of the car. Likewise, the correct solution to the Prisoner’s Paradox hinges on the warden’s inability to divulge Prisoner A’s parole status.
To see this, let’s change the rules of Let’s Make a Deal. Suppose that, once you select Door #1, Monty has the option of revealing either of the remaining doors - even the one hiding a car. We will assume that, if Monty reveals the car, you only have the options of keeping your goat or switching over to the other goat. Will the “switch strategy” still work to your advantage?
To find out, let’s modify our tree diagram and sample space diagram to accommodate Monty’s new-found freedom (see Figure 10 and Figure 11).
Figure 10: Tree diagram for the revised Monty Hall Problem.
Figure 11: Sample space diagram for the revised Monty Hall Problem.
Using the same notations as before, we now have P(D) = 1/3 x 1/2 + 1/3 x 1/2 + 1/3 x 0 = 1/3, P(C) = 1/3, and P(D | C) = 1/2. Hence,
So, if Monty Hall can reveal either of the remaining two doors in every circumstance, then Door #2 has a 50% chance of hiding the car. By the same calculations, Door #1 (the contestant’s door) has a 50% chance of hiding the car, given Door #3 has been opened. Hence, the “switch strategy” does not offer any advantage in this variation.
We have assumed in this variation that Monty Hall flips a coin to decide which door to open. One can imagine a variation in which Monty Hall will reveal the location of car when he can, thus showing the contestant immediately whether he has won or lost. (We will not consider that variation here.)
A recent letter to Ask Marilyn (vos Savant 2003) provides yet another variation on these two problems.
| Suppose I am taking a multiple-choice test. One question has three choices. I randomly choose A. |
| Then the instructor tells us that C is incorrect. Should I switch to B before turning in my paper? |
| Benno Bonke Rotterdam |
Once again, vos Savant gives the correct answer: “It doesn’t matter. You have the same chance either way!”
How does this question differ from the Prisoner’s Paradox and Monty Hall Problem? Conceptually, the difference is that the instructor has no knowledge of which answer you (or your classmates) have selected. In other words, the instructor is not obligated to reveal the (in)correctness of an option different than yours, as Monty Hall and the prison warden are.
Let us confirm vos Savant’s answer. We presume that the teacher will only elect to reveal one of the two incorrect answers. You have selected A. We want to answer the question, what is the probability B is correct, given that C is revealed to be incorrect? Since the instructor’s decision does not depend on your action, we do not require a tree diagram; this is, in fact, a one-stage probability question. The sample space of possible correct answers, from the student’s point of view, consists of three options (A, B, C}, each with a priori probability 1/3. Given the correct answer is not C (denoted C' = {A, B}), we find
Likewise, P(A | C') = 1/2. Thus, given the correct answer is not C, A and B are equally likely to be the correct answer.
Mosteller, F. (1965), Fifty Challenging Problems in Probability with Solutions, Reading, MA: Addison-Wesley.
Rossman, A. and Short, T. (1995), “Conditional Probability and Education Reform: Are They Compatible?” Journal of Statistics Education [Online], 3(2) jse.amstat.org/v3n2/rossman.html
Stansfield, W. (1969), Schaum’s Outline in Genetics, 1st Ed., New York: McGraw-Hill.
Stansfield, W., and M. Carlton (2003), “Bayesian Statistics for Biological Data: Pedigree Analysis,” American Biology Teacher, 66(3).
vos Savant, M. (1996), The Power of Logical Thinking, New York: St. Martin’s Press.
vos Savant, M. (2003), “Ask Marilyn,” Parade Magazine.
Matthew Carlton
Department of Statistics
Cal Poly State University
San Luis Obispo, CA
U.S.A.
mcarlton@calpoly.edu
Volume 13 (2005) | Archive | Index | Data Archive | Information Service | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications