The Role of Statistics Educators in the Quantitative Literacy Movement

Journal of Statistics Education Volume 14, Number 2 (2006), jse.amstat.org/v14n2/jordan.html

Copyright © 2006 by Joy Jordan and Beth Haines all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.

Key Words: Across-the-curriculum approach; Statistics education; Transfer.

Abstract

1. Introduction: The Link Between Statistics and QL

Hence, statistical education should play a critical role in the QL curriculum. The understanding of chance, variability, sampling methods, data analysis, and decision-making are important statistical concepts as well as essential components of QL. Like the QL movement, the statistics education reform pedagogy encourages students to apply statistical and mathematical concepts to interesting, important problems in a variety of contexts. In this sense, statistics is a natural bridge between mathematics and QL; statistics teaches general quantitative principles, yet couches them in a variety of real-world contexts. Furthermore, statistics courses introduce students to sophisticated reasoning and open-ended questions (e.g., what design should be implemented? What analyses should be done?). Statistical training should be focused on conceptual mastery and skill development in a manner that promotes the transfer and application of concepts across contexts, including real-world contexts in which multiple-solution problems predominate. As Lovett (2001) demonstrates in her literature review and empirical work on statistics education, there is still considerable room for improvement when teaching transferable skills. With explicit attention to transfer, statistics courses can successfully foster quantitative literacy in students. Research on transfer demonstrates the importance of providing opportunities for students to practice QL skills and of explicitly showing conceptual connections across contexts. That is, an across-the-curriculum approach is essential to developing QL in students.

Leaders in the QL movement have also consistently recommended a quantitative-reasoning-across-the-curriculum (QRAC) approach to developing quantitative literacy (Cozzens 2003; Madison 2004; Tritelli 2004). A QRAC approach, reflecting the well-established writing-across-the-curriculum model, asks instructors in all disciplines to incorporate explanations of quantitative reasoning and quantitative problem solving in their teaching, and where appropriate, in their course requirements. A QRAC approach reinforces quantitative concepts and problem-solving processes, and allows students to practice and apply skills in new contexts. Furthermore, a QRAC approach often provides opportunities for students to practice skills in areas where they have greater familiarity or expertise, as in their major area. Research on learning (see, for example, Bransford, Brown, and Cocking 2001) and transfer (see, for example, Detterman and Sternberg 1993) has established that familiarity and expertise generally promote more sophisticated reasoning and problem solving. Teaching quantitative reasoning is a natural pedagogical fit in most science and social science courses, and is sometimes a more challenging fit in some humanities and fine arts courses. However, even in the sciences the QRAC approach has not yet become standard practice, and many faculty legitimately struggle with determining the best pedagogies for teaching QL and promoting transfer. In some cases, faculty feel unqualified or simply uninterested in teaching QL. Given that many fields use statistics, statistics educators can provide peer support and help faculty to find a common language to teach quantitative concepts across disciplines.

In this article, we argue that statistics education and statistics educators should play a pivotal role in the QL movement, particularly in helping colleges to incorporate QL across the curriculum. Our argument is based on two sources of information: 1) Our data from statistics courses and other quantitative-intensive courses at Lawrence University and 2) a review of the research literature on transfer of quantitative concepts across contexts. In our view, transfer of quantitative concepts to novel contexts is the greatest challenge to achieving QL. As Douglas Detterman (1993, p.13) concludes in his review of transfer studies, “The surprise is the extent of similarity it is possible to have between two problems without subjects realizing that the two situations are identical and require the same solution.” We believe statistics educators can play a critical role in promoting transferable skills in students and encouraging pedagogical changes and curricular development to support transfer. To this end, in the final section of the article we provide suggestions for the role of statistics educators in the QL movement.

2. Lawrence University Data

2.1 Data on Quantitative-Intensive Courses

Table 1 summarizes the evaluation data according to the level of course: introductory (16 different courses, 787 students), lower (17 different courses, 414 students), or upper (16 different courses, 199 students). Introductory courses have no prerequisites, lower-level courses have at least one prerequisite and are at the 100 or 200 level (i.e. freshman and sophomore level), and upper-level courses are at the 300 or 400 level (i.e., junior and senior level). The quantitative requirement at Lawrence specifies that students take one quantitative-intensive course. Therefore, splitting the data in this way allows us to compare perceptions of students meeting the basic requirement to perceptions of students who elect to take additional quantitative courses. Given that many QRAC programs only require one or two quantitative courses, this comparison will inform our discussion of how statistics educators might facilitate the teaching of transferable skills in the context of such programs.

Looking at Table 1, most of the averages are above 7 (on a 10-point scale), indicating that students tended to evaluate the courses positively, although courses with prerequisites (both lower- and upper-level) were evaluated more positively than courses without prerequisites. Given our focus on QL, we highlight the items most relevant to the development of QL skills: opportunities to develop quantitative reasoning, feedback on quantitative work, learning concepts or skills that can be applied in other courses or that have practical applications, opportunities to explain reasoning, and amount and helpfulness of instruction on quantitative skills. To aid our interpretations, Appendix B summarizes the data by course level, as well as by department.

In terms of opportunities to develop quantitative reasoning and usefulness of feedback on quantitative work, ratings significantly improved as level of course increased. A pleasant exception to the student tendency to rate introductory courses lower was the finding that introductory statistics and introductory computer science were particularly strong in these areas (see Appendix B). Statistics courses with prerequisites (at both the lower- and upper-level) were also rated very highly in terms of opportunities to develop quantitative reasoning (lower-level courses: mean = 8.8, upper-level courses: mean = 9.1) and feedback (lower-level courses: mean = 8.8, upper-level courses: mean = 9.0).

NOTE: ANOVA analyses (by course level) for all questions showed significant results. In pairwise comparisons (using 0.05 as Tukey’s family error rate) the introductory course mean was significantly different from both the lower-level and upper-level means for all questions. The lower-level and upper-level means were only significantly different on questions 2, 4, and 7.

In terms of applications to other courses or practical applications, the means in the introductory courses were significantly lower than the means in the courses with prerequisites. However, when the introductory data were broken down by department, some encouraging differences again emerged. In terms of applications to other courses, students rated statistics (mean = 7.5), computer science (mean = 7.5), and mathematics (mean = 7.4) significantly more highly than other introductory courses (t = 6.86, p-value = 0.00). Similarly, statistics (mean = 7.9) and computer science (mean = 7.9) were rated significantly more highly than other introductory courses in terms of practical applications (t = 8.96, p-value = 0.00). Interestingly, statistics (mean = 9.4) was rated significantly higher than mathematics (mean = 7.2) in terms of practical applications at both the lower (t = 8.09, p-value = 0.00) and upper level (t = 5.59, p-value = 0.00). As shown in Appendix B, this pattern also held for course applications. These findings are consistent with the argument that advanced math is increasingly abstract, whereas statistics emphasizes conceptual application, contributing to QL (Steen 2001).

Ratings of opportunities to explain reasoning also varied significantly with level of course. However, introductory statistics (mean = 7.1) did not stand out as particularly strong, whereas introductory physics (mean = 8.7) was rated strikingly higher than any other introductory course. For courses with prerequisites, statistics was rated highly in terms of explaining reasoning (lower-level courses: mean = 8.9, upper-level courses: mean = 9.0), as were most courses with prerequisites. The differences between introductory and higher-level courses in opportunities to explain reasoning may be accounted for by the smaller class sizes in upper-level courses. Nevertheless, if the goal is to improve QL, these results suggest that an area for possible improvement is to encourage students to monitor and explain their own reasoning at all course levels.

In terms of explicit instruction on quantitative skills and helpfulness of that instruction, introductory courses were rated significantly lower than courses with prerequisites. However, introductory statistics was rated significantly higher than other introductory courses in both the amount (t = 9.07, p-value = 0.00) and helpfulness (t = 6.80, p-value = 0.00) of quantitative instructions, and comparable to ratings of courses with prerequisites. For courses with prerequisites, statistics also stood out as particularly strong in these areas (see Appendix B).

Perhaps not surprisingly, when students were asked to rate the extent to which their overall quantitative skills improved due to a course, introductory courses were rated significantly lower than other courses. However, when asked to assess their initial level of quantitative skills and their skills at the end of the course (questions 14 and 15), the average improvement was significantly greater for students in introductory courses than in other courses. Introductory students seem to recognize that their beginning quantitative skills are at a lower level and that they have improved significantly after the introductory course, yet perhaps they also recognize that there is substantial room for further improvement in their QL skills.

In summary, the data from the course evaluations suggest that students perceive introductory courses less positively than courses with prerequisites. However, statistics stood out as a pleasant exception to this rule in most cases. That is, based on student reports, introductory statistics helped students to learn transferable skills. However, these data provide no basis to assume students will actually go on to use their skills, especially if they are fulfilling the minimal quantitative requirement. A QRAC approach, especially in colleges with minimal quantitative requirements, seems particularly important because it provides opportunities for students to practice learned quantitative skills in other contexts. Furthermore, students fulfilling only the minimal requirement may have less confidence in their quantitative skills, implying that instructors across the curriculum, particularly those in introductory quantitative courses, may need to focus not only on improving students’ quantitative reasoning skills, but also on bolstering students’ confidence and interest in QL. We will explore this point below with attitudinal data from a statistics course.

2.2 Introductory Statistics: Attitudinal Data

NOTE: The confidence subscale includes items 2, 3, 16 (reversed), 22, 27 (reversed), 40 (reversed), and 46 (reversed). The practical utility subscale includes items 10, 11, 15, 19 (reversed), 23 (reversed), 26, 30, 38, 39, 42 (reversed), 44, and 47. The personal growth subscale includes items 1, 12, 20, 24, 25, 28, 32, 43, 45, 49. The interest subscale includes items 4 (reversed), 9, 17, 21 (reversed), and 41.

Overall, students showed significantly improved confidence in their mathematical/statistical abilities. Specific item analyses showed that while 48% of students initially reported nervousness about learning statistics, only 16% reported nervousness on the post-test. In addition, the percentage of students who felt confident about being good at mathematics nearly doubled from pre- (15%) to post-test (29%). Clearly, though, there is still room for improvement.

Student perceptions of the practical utility of mathematics and statistics also significantly improved from pre- to post-test. We were delighted to find that, at post-course, 84% of the students thought statistics helped them to understand the world and 74% noticed familiar statistical concepts in other courses. After taking the introductory statistics course, 90% of students agreed or strongly agreed with the statement, “After I’ve forgotten all the formulas, I’ll still be able to use ideas I’ve learned in statistics.” This understanding of the conceptual applicability of statistical thinking mirrors the attitudinal goals of the QL movement.

Similarly, students were significantly more likely to endorse items about mathematics and statistics contributing to their personal growth on the post-test. For example, at post-course, 74% of students agreed that statistics raises interesting new questions about the world.

Unfortunately, in terms of interest in pursuing further study, there was no significant change from pre- to post-test. We found only 23% of the students wanted to study more statistics. That is, even when students became convinced of the applicability of statistics in other areas and were more confident in their skills, they were reluctant to pursue further study. Garfield and Ahlgren (1994) found similar results in their evaluation of the nationwide Quantitative Literacy Project (66% of students felt statistics was useful, but only 35% wanted to learn more statistics). This reluctance points to a variety of challenges, including the incredible difficulty people have with transferring quantitative skills across contexts (see, for example, Detterman and Sternberg 1993). It should be noted that none of the items on the attitudinal scale measured students’ confidence in actually using statistical skills in the future. Although we saw increased confidence in mathematical and statistical skills following the introductory course, students may feel that transferring and applying those skills in new contexts is a difficult task.

2.3 Summary

3. Review of the Transfer Research Literature: Challenges for Educators

3.1 General Transfer Research

Furthermore, the ability to identify analogous underlying formalisms and concepts is greatly affected by learners’ familiarity with an area. Familiarity affects whether people dig deep and identify underlying principles or get caught in the surface features of the problem. For example, Chi, Feltovich, and Glaser (1981) showed that when physics experts were asked to judge the similarity of physics problems, they did so on the basis of formulaic or structural similarity (e.g., problems that involve torque). In contrast, novices judged similarity based on the surface structure of the problems (e.g., mention of an inclined plane). Similarly, Reed and Evans (1987) found that college students did more poorly on acid-concentration mixture problems than on a more familiar temperature-of-water mixture task, even though the problems were isomorphic, requiring the same weighted-average principle for solution. More encouragingly, college students who were trained with the temperature prediction problems were able to apply the principle to solve the acid-concentration mixture problems, if given the simple hint that the principles were the same in both cases. Because of students’ lower levels of familiarity and expertise within many areas, we hypothesize that promoting transfer will necessitate explicit instruction on recognizing the underlying formalisms and concepts in problems, using a common quantitative language across disciplines.

Other issues involved in understanding transfer include a) the degree of similarity between the context in which a principle is learned and the new context in which it should be applied, and b) the amount of time between initial learning and subsequent application. Barnett and Ceci (2002) offer a taxonomy for judging the extent to which far transfer (i.e., transfer to dissimilar contexts) occurs. They demonstrate in their review that very little of the lab research on transfer has used stringent transfer tests that require the kind of flexible, sophisticated reasoning and application of principles over a long period of time that educators desire and QL requires. Much educational research is also guilty of looking only at the transfer of isolated concepts on tests that occur very near the time of training (Lovett and Greenhouse 2000).

From an educational perspective, one productive line of research explores the cumulative effects of disciplinary training on QL. For example, Lehman and Nisbett (1990) examined the effects of disciplinary undergraduate training on conditional reasoning, and statistical and methodological reasoning. They found that social science training led to substantial improvements in statistical and methodological reasoning, whereas natural science and humanities training produced improvements in conditional reasoning and smaller effects on statistical reasoning. This research provides an encouraging start for a QRAC program and also points out the importance of identifying the QL concepts students should have in their repertoire and in what areas of the curriculum those concepts are being taught. However, this research does not address the specific pedagogies and experiences that lead to improvement. Consequently, the mechanisms necessary for fostering transfer of concepts across contexts still need to be ascertained.

Statistics education research has begun to describe such mechanisms by identifying pedagogical techniques that promote student learning (Garfield 1995; delMas, Garfield, and Chance 1999). Some important educational implications have emerged from this work. For example, delMas, et al. (1999) demonstrated that helping students to explicitly identify and confront their statistical misconceptions led to better statistical reasoning about sampling distributions when working with computer simulation demonstrations. Research in math education also supports this conclusion. Shaughnessy (1977) highlighted the importance of students articulating their misconceptions in learning finite mathematics through activity-based pedagogy. This research implies that teaching students to metacognitively monitor their reasoning process and thereby identify possible misconceptions is an integral part of promoting QL.

Students’ tendency to categorize concepts they have learned as either domain-specific or as broadly applicable presents another challenge in promoting transfer. Bassok and Holyoak (1989) gave high school and college students training in solving isomorphic algebra (arithmetic progression word problems) problems or physics (motion in a straight-line with constant acceleration) problems. After training in one domain, they looked at transfer to the other domain. They found that algebra training, even if it involved training using word problems in a specific domain (e.g., money: salary increases or loans; motion, i.e., physics problems of bodies moving in a straight line with constant acceleration), led to successful application of the principles to physics problems. However, training in physics, with physics materials and units, did not lead to the solving of isomorphic algebra problems. They interpreted the findings as reflecting the students’ beliefs that algebra is content free and can be applied across a variety of domains. Physics, on the other hand, is seen as more content bound—specified units are presented, leading students to conclude that physics equations are constrained to particular domains (e.g., motion concepts). Statistics educators can help address this problem of students too narrowly defining concepts. By leading across-the-curriculum discussions, statistics educators can help faculty to identify quantitative concepts taught in their disciplines and to create a shared language for describing these concepts.

A final obstacle in promoting transfer lies in understanding the extent to which the problem-solving context triggers reasoning biases. Reasoning biases occur for a variety of reasons not limited to but including invalid intuitive understanding based on practical experience, a tendency to favor evidence that confirms one's own hypotheses, and a tendency to under- or overestimate or ignore population base rates based on one's own limited experience. The extensive research on reasoning biases (e.g., Kahneman, Slovic, and Tversky 1982; Klaczynski and Gordon 1996) highlights the importance of determining not only what promotes transfer but also what may interfere with it. For example, the representativeness heuristic involves judging the probability of some event on the basis of the extent to which the event represents or resembles the expected and intuitively incorrect outcome, instead of considering the prior probability of outcomes. In their classic work, Tversky and Kahneman (1982a) asked people to judge the probability that someone was a librarian, farmer, salesman, pilot, or physician given the following description, “Steve is very shy and withdrawn, invariably helpful, but with little interest in people, or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.” People routinely estimated the probability that Steve is a librarian to be quite high, no doubt based on their stereotypes, and ignored the relevant base-rate information they were given (i.e., there were many more farmers than librarians) that should enter into any reasonable estimate. When not given the personality description, participants did use base-rate information correctly. Research on reasoning biases suggests that people are more likely to draw on statistical principles when making objective decisions or dealing with explicitly quantitative information than when making decisions about equally uncertain social situations or dealing with less easily quantified information. Statistics educators must help students to attend to base rates and quantitative information even when the linguistic demands in the problem encourage more subjective assessments.

Another interesting bias, the self-serving and confirmatory bias, refers to the tendency to treat evidence consistent with one’s beliefs or goals more favorably than inconsistent evidence (see Haines and Moore 2003 for a review). Klaczynski and Gordon (1996) found that students judged goal-enhancing evidence to be more convincing than neutral or goal-threatening evidence, but interestingly, they were more likely to use statistical reasoning when presented with goal-threatening problems. That is, threatening evidence seemed to stimulate more sophisticated statistical reasoning. The good news is that transfer is more likely when people are engaged or feel their beliefs are challenged, and the bad news is that people require little empirical evidence if their beliefs are confirmed. Statistics educators can play a central role in encouraging students to routinely examine the quantitative evidence.

3.2 Statistics Transfer Research

Kosonen and Winne (1995) extended the work of Fong and colleagues. They trained groups of undergraduates on concepts associated with sampling and the law of large numbers as well as on what kinds of reasoning errors to avoid (truism, egocentric bias based on their own experience, attribution to character’s disposition and under-using objective data, and speculation—adding data to the problem or generating a hypothesis not founded on the problem’s description). They found that training on avoiding biases helped students with the kinds of reasoning problems where they had the most difficulty (e.g., using base-rate information rather than dispositions when making judgments). The researchers speculated that the success of this brief training “can be attributed, in part, to statistical heuristics that were already but inadequately part of the students’ repertoires for reasoning” (Kosonen and Winne 1995, p. 44). Hence, students may develop and use statistical heuristics, some correct and some incorrect, before entering a formal statistics class. A key finding from Bransford, et al. (2001, pp. 14-15) synthesis of the learning literature is the importance of addressing students’ preconceptions: “If their initial understanding is not engaged, they may fail to grasp the new concepts and information that are taught, or they may learn them for purposes of a test but revert to their preconceptions outside the classroom.” Consequently, correcting and refining student intuition may, in fact, be more productive than simply teaching a concept from scratch. These research findings are encouraging because they demonstrate that students can be successfully trained to avoid reasoning biases.

Lovett (2001) did a series of studies on college students’ reasoning about statistics using an approach that integrates cognitive theory as well as methods and findings from applied research on statistics education. Her team drew on cognitive theory to model how students learn statistical reasoning, and then tested the model empirically using both laboratory and statistics classroom-based designs. This approach allowed them to identify areas where statistics students showed good mastery (e.g., interpreting descriptive and inferential statistics), and diagnose specific areas of difficulty (e.g., choosing appropriate statistical approaches and drawing conclusions from statistical analyses). They used this information to design a computerized learning environment to help students overcome these difficulties. Students, even those with no prior statistical training, showed substantial improvement after a brief (45 minute) computerized interactive session, especially if they received specific feedback throughout their problem-solving attempts (as opposed to feedback at the end). Lovett’s research demonstrates that detailed analysis of student learning can be used to successfully and specifically modify learning environments to provide practice and guidance in difficult areas, thereby helping to establish more comprehensive, transferable skills.

3.3 Summary and Implications

4. Suggestions for Statistics Educators

• Statistics educators can lead discussions to identify key QL concepts and create a shared vocabulary for labeling those concepts. Because statistics is used in so many fields and statisticians routinely consult with colleagues across the curriculum, statisticians are uniquely poised to articulate how similar statistical concepts are used in different contexts. Unless educators create a shared language and explicitly label concepts, the transfer research shows that students’ lack of familiarity with a discipline is likely to impede sophisticated quantitative reasoning. Similarly, Lovett (2001) emphasizes the importance of creating a common language for statistics concepts across disciplines. Utts (2003) and Garfield (1995) have begun the task of identifying key statistical concepts to include in the QL repertoire.

• Statistics educators can teach students metacognitive skills for monitoring and verbalizing their reasoning processes. The statistics reform movement advocates active learning and the use of real data, yet discovery exercises alone do not always lead to a deep conceptual understanding that transfers successfully to new contexts. As our data demonstrate, introductory students often have fewer opportunities to articulate their reasoning, which may impede the development of deep conceptual understanding. Statistics teachers can help students derive the underlying concepts and principles, rather than focusing exclusively on the surface features of the problem. If this is done across the curriculum, students will have the opportunity to acquire the metacognitive skills and the language to articulate their own reasoning and conceptual understanding. Assessment data will be needed on actual QL skills to determine whether developing metacognitive skills actually promotes transfer.

• Statistics educators can teach students to do a bias check when solving quantitative problems. The research on reasoning biases and decision-making clearly establishes that everyone is prone to biases, like the representativeness heuristic and confirmation bias. Besides instruction on correct problem-solving approaches, students need tools to check for reasoning traps and faulty intuitions that could lead them astray.

• Statistics educators can help students to establish positive attitudes about quantitative problem solving and hence increase confidence about QL across disciplines. In our pre-post assessment of introductory statistics students’ attitudes, two findings consistent with the math anxiety literature emerged and simultaneously provided encouragement that this anxiety can be conquered. First, in the pre-test, students rated themselves as not particularly good at mathematics. Their ratings significantly improved at the conclusion of the course. Student ratings of the extent to which statistics made them nervous also significantly improved following the introductory course. Although the mechanisms for the improvement are not clear, the data suggest that statistical education can help to build student confidence and improve attitudes about statistics.

• Statistics educators can develop pedagogies that include more opportunities with open-ended problems that have multiple solutions, allowing students to focus more on the reasoning process than on getting a particular right answer. If students are encouraged to compare and contrast multiple problems and multiple solution paths, they are more likely to acquire deeper, and consequently more transferable, conceptual understanding of the underlying principles (Lovett and Greenhouse 2000). Data-driven projects, like those now used in statistics courses, may be a useful way for students to work with open-ended problems

• Statistics educators can teach faculty how to assess the effectiveness of pedagogies used to teach quantitative skills. The recent research from the statistics education reform movement provides an excellent model for designing and interpreting classroom research (see, for example, delMas, et al. 1999). We recommend assessing students’ understanding of statistical principles and willingness to engage in quantitative problem solving in new contexts, as well as their metacognitive ability to recognize when statistical reasoning is needed and to evaluate the validity of their own reasoning processes.

• Statistics educators can provide a support network for faculty interested in incorporating quantitative reasoning into their courses. Besides stimulating a general dialogue about QL goals across the curriculum, statistics teachers can offer individual consultation to help faculty develop particular pedagogies for teaching QL in their discipline. Ideas from the statistics reform movement (e.g., active learning, use of real data) can be extended and applied to other disciplines. These types of individual consultations create an opportunity for faculty who do not typically teach in a quantitative area to address any concerns they have and to articulate their own questions about quantitative concepts.

Appendix A — Student Evaluation Form for Quantitative-Intensive Courses

Course Title & Number: ______________________ Term/Year: _________

1. What types of quantitative work did you do in this class?

	Symbolic proofs?		Never		Sometimes	  Frequently
	Statistical analysis of data? 	Never		Sometimes	  Frequently
	Interpretation of graphs?	Never		Sometimes	  Frequently
	Problem solving?		Never		Sometimes	  Frequently
	Computer programming?		Never		Sometimes	  Frequently
	Other _____________________________________________

2. Usefulness of feedback on quantitative work
	1	2	3	4	5	6	7	8	9	10
    Not Useful		  	   Somewhat				   Very Useful

3. Did you receive feedback on quantitative work other than on quizzes or exams? 
		No 		Yes   On what? _____________________________ 	  

4. Opportunities to develop quantitative reasoning (for example, but not limited to, analyzing 
   evidence, detecting fallacies in reasoning, questioning assumptions and conclusions)
	1	2	3	4	5	6	7	8	9	10
     None			    Some			   	     Many

5. Did you learn concepts or quantitative skills that you will apply in other courses?
	1	2	3	4	5	6	7	8	9	10
  Not at all			    Some 			   	     Many

If so, please give examples: 

6. Did you learn concepts or quantitative skills that have practical applications?
	1	2	3	4	5	6	7	8	9	10
Not at all			    Some				     Many

If so, please give examples:

7. How often were you asked to explain the reasoning behind your work?
	1	2	3	4	5	6	7	8	9	10
    Never			  Sometimes				   Very Often

8. Extent to which your overall quantitative skills improved due to this course
	1	2	3	4	5	6	7	8	9	10
   Not at all			   Somewhat 				   Very Much 

9. Opportunities for individual help on your quantitative skills
	1	2	3	4	5	6	7	8	9	10
     None			    Some			   	     Many

10. Clarity of guidelines and expectations for quantitative work
	1	2	3	4	5	6	7	8	9	10
   Not Clear			   Somewhat			   	   Very Clear

11. Amount of explicit instruction on quantitative skills  	
	1	2	3	4	5	6	7	8	9	10
    None			    Some			   	   Very Much

12. Helpfulness of the instruction on quantitative skills?
	1	2	3	4	5	6	7	8	9	10
   Not Helpful		   	   Somewhat 				  Very Helpful

13. Please comment on how this course has helped you to develop your quantitative skills.  What 
aspects of the course were particularly helpful?















14. Did you use the quantitative tutoring services of the Center for Teaching and Learning?
 	 	Yes		No

	If yes, how often?

15. To what extent did you find the assistance helpful?
	1	2	3	4	5	6	7	8	9	10
  Not at all			   Somewhat 				  Very Helpful

16. Rate your quantitative skills at the beginning of the course.
	1	2	3	4	5	6	7	8	9	10
     Poor			   Average				    Excellent

17. Rate your quantitative skills at the end of the course.
	1	2	3	4	5	6	7	8	9	10
     Poor			   Average				    Excellent

Appendix B — Course Evaluation Data, Summarized by Department and Level of Course

Appendix C—Dartmouth Attitudinal Survey Items (Korey 2000)

1. To understand mathematics I sometimes think about my personal experiences.
2. I am good at mathematics.
3. If I work at it, I can do well in mathematics.
4. Most subjects interest me more than mathematics.
5. Mathematics is essentially an accumulation of facts, rules, and formulas to be memorized and used.
6. Good mathematics teachers show students the exact way to answer the questions they’ll be tested on.
7. Using a computer makes learning mathematics more complicated than it needs to be.
8. People who are good at mathematics can do mathematics quickly.
9. I enjoy learning new things in mathematics.
10. Mathematics helps me understand the world around me.
11. Mathematics has been an important tool to help me learn other subjects.
12. I have taken some mathematics courses in high school and college that were taught in a very interesting way.
13. For me, mathematics rarely involves exploration, investigation, or experimentation.
14. I like exploring problems using real data and computers.
15. Many situations in the world around me can be modeled mathematically.
16. I often feel like I’m missing something important in mathematics class.
17. I want to study more mathematics.
18. Working in groups helps me learn mathematics.
19. I rarely encounter situations that are mathematical in nature outside of school.
20. Doing mathematics helps me understand myself.
21. I try to avoid courses that involve mathematics.
22. When I get stuck on a mathematics problem, I can usually find my way out.
23. Becoming more proficient in mathematics prepares you for the next mathematics class, but that’s about all.
24. Writing about mathematics makes it easier to learn.
25. In mathematics you can be creative and discover things for yourself.
26. After I’ve forgotten all the formulas, I’ll still be able to use ideas I’ve learned in mathematics.
27. I’m never sure my answer is right until I’m given the solution.
28. Doing mathematics raises interesting new questions about the world.
29. Learning mathematics makes me nervous.
30. I often see familiar mathematical concepts in courses outside of mathematics.
31. Doing mathematics helps me think clearly and logically.
32. Mathematical thinking helps me make intelligent decisions about my life.
33. I don’t really understand mathematics until I work it out for myself.
34. Expressing scientific concepts in mathematical equations just makes them more confusing.
35. I don’t need a good understanding of mathematics to achieve my career goals.
36. Good statistics teachers show students the exact way to answer the questions they’ll be tested on.
37. Using a computer makes learning statistics more complicated than it needs to be.
38. Statistics helps me understand the world around me.
39. Statistics has been an important tool to help me learn other subjects.
40. I often feel like I’m missing something important in statistics class.
41. I want to study more statistics.
42. I rarely encounter situations that are statistical in nature outside school.
43. In statistics you can be creative and discover things for yourself.
44. After I’ve forgotten all the formulas, I’ll still be able to use instead I’ve learned in statistics.
45. Doing statistics raises interesting new questions about the world.
46. Learning statistics makes me nervous.
47. I often see familiar statistical concepts in courses outside of statistics.
48. Doing statistics helps me think clearly and logically.
49. Statistical thinking helps me make intelligent decisions about my life.
50. I don’t really understand statistics until I work it out for myself.
51. I don’t need a good understanding of statistics to achieve my career goals.

References

Bassok, M. and Holyoak, K.J. (1989), “Interdomain Transfer Between Isomorphic Topics in Algebra and Physics,” Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 153-166.

Bransford, J.D., Brown, A.L., and Cocking, R.R. (eds.) (2001), How People Learn: Brain, Mind, Experience, and School, National Academy Press.

Chi, M.T.H., Feltovich, P.J., and Glaser, R. (1981), “Categorization and Representation of Explanations: How Students Study and Use Examples in Learning to Solve Problems,” Cognitive Science, 13, 145-182.

Cozzens, M.B. (2003), “Educational Policy and Decision Making,” in Madison and Steen (eds.), Quantitative Literacy: Why Numeracy Matters for Schools and Colleges (pp. 197-198), National Council on Education and the Disciplines.

delMas, R.C., Garfield, J., and Chance, B.L. (1999), “A Model of Classroom Research in Action: Developing Simulation Activities to Improve Students’ Statistical Reasoning,” Journal of Statistics Education [Online], 7(3).
jse.amstat.org/secure/v7n3/delmas.cfm

Detterman, D.K. (1993), “The Case for the Prosecution: Transfer as an Epiphenomenon,” in Detterman and Sternberg (eds.), Transfer on Trial: Intelligence, Cognition, and Instruction (pp 1-24), Norwood, New Jersey: Ablex Publishing.

Detterman, D.K. and Sternberg, R.J. (eds.) (1993), Transfer on Trial: Intelligence, Cognition, and Instruction, Norwood, New Jersey: Ablex Publishing.

Fong, G.T., Krantz, D.H., and Nisbett, R.E. (1986), “The Effects of Statistical Training on Thinking about Everyday Problems,” Cognitive Psychology, 18, 253-292.

Garfield, J. (1995), “How Students Learn Statistics,” International Statistical Review, 63, 25-34.

Garfield, J. and Ahlgren, A. (1994), “Student Reactions to Learning About Probability and Statistics: Evaluating the Quantitative Literacy Project,” School Science & Mathematics, 94(2), 89-95.

Haines, B. A., and Moore, C. F. (2003), “Integrating Themes from Cognitive and Social Cognitive Development into the Study of Judgment and Decision Making,” in Schneider and Shanteau (eds.), Emerging Perspectives on Judgment and Decision Research (pp. 246-283), New York: Cambridge University Press.

Kahneman, D., Slovic, P., and Tversky, A. (eds.) (1982), Judgment Under Uncertainty: Heuristics and Biases, Cambridge University Press.

Klaczynski, P.A. and Gordon, D.H. (1996), “Everyday Statistical Reasoning During Adolescence and Young Adulthood: Motivational, General Ability, and Developmental Influences,” Child Development, 67, 2873-2891.

Konold, C. (1989), “Informal Conceptions of Probability,” Cognition and Instruction, 6(1), 59-98.

Korey, J. (2000), “Dartmouth College of Mathematics Across the Curriculum Evaluation Summary: Mathematics and Humanities Courses,” Mathematics Across the Curriculum at Dartmouth College,
www.math.dartmouth.edu/~matc/Evaluation/humeval.pdf

Kosonen, P. And Winne, P.H. (1995), “Effects of Teaching Statistical Laws on Reasoning About Everyday Problems,” Journal of Educational Psychology, 87(1), 33-46.

Lehman, D.R. and Nisbett, R.E. (1990), “A Longitudinal Study of the Effects of Undergraduate Training on Reasoning,” Developmental Psychology, 26(6), 952-960.

Lovett, M.C. (2001), “A Collaborative Convergence on Studying Reasoning Processes: A Case Study in Statistics, in Carver and Klahr (eds.), Cognition and Instruction: Twenty-five Years of Progress (pp. 347-384), Mahwah, NJ: Erlbaum.

Lovett, M.C. and Greenhouse, J.B. (2000), “Applying Cognitive Theory to Statistics Instruction,” The American Statistician, 54(3), 196-206.

Madison, B.L. (2004), “Two Mathematics: Ever the Twain Shall Meet?,” Peer Review, 6(4), 9-12.

Reed, S. and Evans, A. (1987), “Learning Functional Relations: A Theoretical and Instructional Analysis,” Journal of Experimental Psychology: General, 116, 106-118.

Scheaffer, R.L. (2003), “Statistics and Quantitative Literacy,” in Madison and Steen (eds.), Quantitative Literacy: Why Numeracy Matters for Schools and Colleges (pp. 145-152), National Council on Education and the Disciplines.

Shaughnessy, M.J. (1977), “Misconceptions of Probability: An Experiment with a Small-Group, Activity-Based, Model Building Approach to Introductory Probability at the College Level,” Educational Studies in Mathematics, 8, 285-316.

Steen, L. (ed.) (2001), Mathematics and Democracy: The Case for Quantitative Literacy, Princeton, NJ: The National Council.

Tritelli, D. (2004), “From the Editor,” Peer Review, 6(4), 3.

Tversky, A. and Kahneman, D. (1982a), “Judgment Under Uncertainty: Heuristics and Biases,” in Kahneman, Slovic, and Tversky (eds.), Judgement Under Uncertainty: Heuristics and Biases (pp. 3-20), Cambridge University Press.

­--- (1982b), “Evidential Impact of Base Rates,” in Kahneman, Slovic, and Tversky (eds.), Judgement Under Uncertainty: Heuristics and Biases (pp. 153-160), Cambridge University Press.

Utts, J. (2003), “What Educated Citizens Should Know About Statistics and Probability,” The American Statistician, 57(2), 74-79.

Joy Jordan
Department of Mathematics
Lawrence University
Appleton, WI 54912
U.S.A.
joy.jordan@lawrence.edu

Beth Haines
Department of Psychology
Lawrence University
Appleton, WI 54912
U.S.A.
beth.a.haines@lawrence.edu