Link to Lawrence Lesser's letter: http://jse.amstat.org/v11n3/lesser_letter.html

Martin, M. A. (2003), "'It’s like ... you know': The Use of Analogies and Heuristics in Teaching Introductory Statistical Methods," *Journal of Statistics Education* [Online], 11(2).
(jse.amstat.org/v11n2/martin.html)

I appreciate Larry Lesser's letter outlining some additional analogies to supplement those described in my article "It’s like ... you know': The Use of Analogies and Heuristics in Teaching Introductory Statistical Methods." I thoroughly agree with his comment that the search for a more complete set of analogies is important and ongoing, and the analogies he describes are, indeed, evocative and useful. The purpose of my response to Professor Lesser's letter is twofold: to offer some brief commentary on the analogies he describes; and to offer two new analogies of my own.

The first analogy described by Professor Lesser, the extension to the "crime-and-punishment" view of hypothesis testing, is certainly evocative, but like most analogies linking hypothesis testing to real-world experience, great care is needed not to overplay the analogy. For example, the idea of "narrowing a search", suggesting refinement based on evidence, runs counter to the notion in statistical testing that the choice of a one- or two-sided alternative should be made prior to examining the data, lest the tester run the risk of being accused of "data snooping". An analogy I prefer in explaining the use of one- versus two-tailed rejection regions is the notion of "putting all one's eggs in one basket". If one believes, a priori, that a test against a one-sided alternative is appropriate, then it makes sense to put all one's eggs (potential for Type 1 error) in a single basket (that is, on only one side), rather than to hedge one's bets (as in a two-sided alternative) by splitting one's eggs (Type 1 error) across two baskets (sides).

I very much liked the idea of using the "leaves falling from trees" image to describe confidence intervals, though again care must be taken not to continue the analogy too far. I do not think, for instance, that invoking the idea of a neighboring tree is productive, as it may lead students to think that if two piles overlap substantially, one could conclude that the two trees must be at the same location. Of course, one cannot conclude that two populations' means are the same just because individual confidence intervals overlap, even if the overlap is substantial. To the extent that interval estimation and hypothesis testing apparently defy the laws of logic (sequential decisions cannot necessarily be combined to make the obvious conclusion, at least at a specified level of significance), physical analogies often fail because the relationships between their elements usually do obey simple logical rules. Perhaps there is a role here for counterintuitive examples to encourage students to look beyond simplistic ideas.

We must also keep in mind that cultural issues make some analogies less universally useful than others. The tree analogy is unlikely to be as evocative to classes in tropical climes where trees do not shed their leaves. Likewise, objects like merry-go-rounds (or for that matter see-saws) may not be in the common experience of students outside Western cultures. The usefulness of the legal analogy for hypothesis testing is enhanced by the fact that the American judicial system is so well-known thanks to its depiction in the mass media, but it may be rather less useful in countries with vastly different legal systems. Analogies based on sports such as basketball, are likely to work well in North America, but less well in the United Kingdom. Similarly, an analogy based on cricket might work well in the UK but be almost useless in the USA. Sports analogies are particularly difficult to use because they not only fail to cross cultural divides but, more importantly, the gender divide. I learnt an enormously valuable lesson when my use of the baseball World Series in an example of a probability calculation caused bewilderment for over half the class, notably for students from other cultures and for many female students as well as a few male students who had no interest in or knowledge of baseball. Suffice to say that baseball examples no longer figure in my teaching (particularly now that I teach in Australia). Physical analogies such as the liquids analogy put forward by Freedman et al transcend cultural barriers and so are very reliable. Cultural blocks are particularly hard to predict and defeat, particularly when modern classrooms represent such a rich blend of cultures and experiences. This problem can best be addressed through the development of as rich a set of analogies, examples and related stories as possible.

I close my response to Professor Lesser's letter with a couple of analogies I have recently started to use in my own teaching. The first is rather general, but the reaction I have had from students has been very positive. The analogy grew out of a desire I had to replace the somewhat sterile description of statistics I had consistently found at the front of most textbooks, namely, that statistics was the "science of obtaining, describing and analyzing data". This description belied, for me, the excitement and importance of our discipline, so initially I replaced this "textbook" definition with the more inviting idea that "statistics is humanity's reaction to randomness". Thinking further about this characterization, I found myself thinking about statistics as being about taking the vast amount of information available today and compressing it in a way that made it digestible and immediately useful. My simple analogy for the field became that statistics was akin to "information compression", the process of distilling useful information out of potentially huge lists of numbers. Of course, statistics is more than this! But I have found that this very broad analogy has encouraged my students to think of statistics as an essential part of understanding all the information that constantly bombards them in their daily lives. Moreover, the idea of compression is very topical since these days students are very familiar with the idea that "information" they download from the internet is usually compressed to speed transmission. Indeed, the notion of "lossless compression" has a statistical analog in the concept of sufficiency. Of course, the analogy between the discipline of statistics and the idea of information compression is far too broad to be useful as anything but a motivational tool. Nevertheless, it is an attractive idea, one that draws a parallel with an experience from most students' lives, particularly if they have used the internet.

Finally, I describe an analogy I have found very useful in describing stepwise model selection techniques such as forward selection, backwards elimination and forward stepwise regression. The analogy assists students to see these techniques as much more than "black boxes" that reveal a good model. The analogy is that of a parent dispensing candy from a bag containing numerous different types of candy to a child. Each piece of candy is analogous to a covariate, and the strategy adopted by the parent corresponds to the particular statistical method of model building. For example, the forward selection method begins with the parent holding the bag and offering the child one piece of candy at a time. The child, not knowing how many pieces they will be offered, selects the best single piece of candy and eats it. At each step, the parent continues to offer another piece of candy, and the child chooses the best remaining piece of candy (bearing in mind its taste combination with what has already been ingested). The process continues until the child, having eaten enough candy, refuses any piece left in the bag (this may happen either because the child is sated or because the remaining pieces are just not to the child's liking). Note that the fact that the child eats the piece of candy at each step prevents the selection process from ever going backwards. The final "model" is the collection of candy (variables) selected by the child.

The analogy for backwards elimination begins with the child holding the bag, and the parent insisting that the child return at least some of the candy, one piece at a time, until the parent is satisfied that what remains will not cause the child to be sick. In this case, the child first returns the single piece of candy they like the least, then the second worst piece of candy, and so on. The process ends when the parent relents and allows the child to keep what remains in the bag after sufficiently many steps. To enforce the "elimination" part of the algorithm, at each step, the parent ingests the returned piece of candy.

The analogy for forward stepwise regression begins, like forward selection, with the parent holding the bag. Now, the child is offered candy one piece at a time, but is told not to eat anything until the entire process is complete. After making each selection, the child is asked whether they would like to return a piece they had previously selected, the choice presumably based on the taste combination of the group selected at a particular time as well as on the child's judgement of the likelihood of being offered another piece. The process ends when each of the child and parent agree that both the child has the candy combination they most desire and that the child does not have so much that they will be ill.

The analogies work well because each step of the algorithms in question has an easily understood equivalent in the stories. The situation is a familiar one: students have all been in situations where they must select a sequence of goods, and the decision process guiding such accumulations is easy to visualize. A weakness of the analogy is that in the story, the choices are made in what appears a highly subjective way, while in the model selection context, choices are made in a more objective way. This difference needs to be made clear at the outset. But most importantly, students can easily see the motivation for each of the three techniques, and they can more easily understand how three such techniques might result in three reasonably different collections of candy (models) as a final answer.

Again, I appreciated Professor Lesser's comments, and I am grateful for being offered the opportunity to respond.

Michael A. Martin

Australian National University

Canberra, ACT 0200

Australia

Michael.Martin@anu.edu.au

Volume 11 (2003) | Archive | Index | Data Archive | Information Service | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications