M. Leigh Lunsford
Longwood University
Ginger Holmes Rowell
Middle Tennessee State University
Tracy Goodson-Espy
Appalachian State University
Journal of Statistics Education Volume 14, Number 3 (2006), jse.amstat.org/v14n3/lunsford.html
Copyright © 2006 by M. Leigh Lunsford, Ginger Holmes Rowell and Tracy Goodson-Espy all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.
Key Words:Action Research
Though our class sizes were relatively small, which is not uncommon in post-calculus probability and statistics courses at many colleges and universities, we hope our results will be none the less interesting to other instructors of these courses. In summary, we concluded that we could not necessarily expect our post-calculus introductory-level probability and statistics students to have good graphical interpretation and reasoning skills concerning sampling distributions even if they appeared to understand the basic theory and were able to perform computations using the theory. We also found that the ability to recall facts about sampling distributions did not imply an ability to apply those facts to solve problems, and thus, our students needed to practice with concepts in order to develop proficiency. In addition, we believed that some of our students confused the limiting result about the shape of the sampling distribution (i.e. as n increases the shape becomes approximately normal, via the CLT) with the fixed (i.e. non-limiting) result about the magnitude of the variance of the sampling distribution, regardless of its shape (i.e. for random samples of size n the variance of the sampling distribution is , via mathematical expectation, where is the population variance). Lastly, we discovered that the use of computer simulation, only for demonstration purposes, was not sufficient for developing deep graphical understanding of concepts associated with sampling distributions and the CLT. Thus, in our future classes, we will combine these simulations with well-designed activities or assignments. This paper provides the details of our methods and results along with some changes we intend to implement which will, hopefully, improve the teaching of sampling distributions and the CLT in our post-calculus probability and mathematical statistics course.
For our classroom research, we initially analyzed the data by considering our instructional goals during the lessons under analysis. Following our initial analyses, we examined the students’ responses and looked for possible explanations for their responses beyond our instructional goals. Student surveys and interviews were also collected. As these survey and interview results were not shared with the class instructor until after class grades were posted, students were encouraged to be open and honest in their reactions to class activities. Ultimately, our goal was to use our results to inform and improve our teaching of this material. However, we hope that others who teach probability and statistics courses will find our results interesting and useful.
In teaching sampling distributions and the CLT in the Math 300 course, we used a traditional text (Hogg and Tanis 2001) along with a computer simulation called Sampling SIM (delMas 2002), and a corresponding activity, Sampling Distributions and Introduction to the Central Limit Theorem, which was slightly modified from one provided by Rossman, et al. (1999) and is an earlier version of an activity by Garfield, delMas, and Chance (2000). Please see Appendix A for a copy of the activity. After an initial in-class demonstration of Sampling SIM, we assigned the activity as an out-of-class group project for which the students turned in a written report. This typed report was a technical document in which students were required to clearly state the problems clearly, explain their approaches, and submit their solutions and conclusions. In order to incorporate more activities into our Math 300 course (as part of our A&I grant), we omitted most of the chapter on Multivariate Distributions (Hogg and Tanis 2001, chap. 5). The coverage of sampling distributions and the CLT occurred near the end of the semester; however, we spent more time on these topics than when we had previously taught the course. This included exposing our students to an application of the CLT by covering the sections of the textbook dealing with confidence intervals and sample sizes for proportions (Hogg and Tanis 2001, sections 7.2, 7.5, and 7.6). These sections were not typically covered in our Math 300 course.
We taught the Math 400 course principally with a lecture method using the same text as Math 300. As with most second courses in the typical post-calculus probability and statistics sequence, the topics covered included an overview of probability topics (including the CLT) and coverage of standard statistical topics (Hogg and Tanis 2001, chapters 7 and 8). We used simulations, including Sampling SIM and simulations from the Virtual Laboratories in Probability and Statistics (VLPS) (Siegrist 1997), exclusively for in-class demonstrations. While we did use the statistical software package, Minitab, for computations, in contrast to the Math 300 course, we did not use any activities in-class or out-of-class nor did we incorporate simulations into any homework assignments. The same professor taught both courses.
From the quantitative assessment tool we collected information about student learning of the sampling distribution and CLT topics via pretest and posttest scores. In addition to seeing how our results compared to previous studies (delMas, et al. 1999a, 1999b, 1999c, 2002), we explored the data to try to describe our students’ understanding and reasoning. We also wanted to determine if there were any noticeable differences in the assessment results of our second semester students versus our first semester students. Thus we examined student responses on the quantitative assessment tool to extract trends and examine reasoning skills that might explain our student responses. Details of these results are given in the next section of this paper.
Total Number of Questions = 27 | Average Number of Correct Answers (Percent Correct) | Standard Deviation in Number (Percent) |
---|---|---|
Math 300 Pretest (N = 18) | 9.9 (36.6%) | s = 3.20 (11.8%) |
Math 300 Posttest (N = 18) | 18.5 (68.5%) | s = 4.26 (15.8%) |
Math 300 Paired Difference Post - Pre (N = 18) | 8.6 (31.8%) | s = 5.45 (20.2%) |
Math 400 Pretest (N = 7) | 12.2 (45.0%) | s = 3.81 (14.1%) |
Math 400 Posttest (N = 7) | 18.9 (69.9%) | s = 2.97 (11.0%) |
Math 400 Paired Difference Post - Pre (N = 7) | 6.7 (24.9%) | s = 2.69 (20.0%) |
As expected, we did see significant improvement in student performance from pretest to posttest in both classes. Even with our small sample sizes the paired differences of posttest minus pretest scores were significantly greater than zero for the Math 300 class (t(17) = 6.7, p < 0.0001) and for the Math 400 class (t(6) = 6.6, p = 0.0003). However, while we did see improvement, we were disappointed with the low percentage of correct responses for these mathematically inclined students on the posttest. We were also somewhat surprised to see that the average percent correct on the posttest was not very different between the Math 300 and Math 400 students. This could be a function of several factors including the types of students enrolled in the two courses and when the tests were administered. For the Math 300 students the pretest and posttest were given within a three week period of study of sampling distributions and the CLT. As previously mentioned, the Math 400 students were tested at the beginning and end of the semester, during which time a wide variety of topics were taught. Thus the Math 300 students’ scores may be artificially high due to their recent coverage of the material. It is also interesting to observe that the average score on the Math 400 pretest (given at the beginning of the semester) shows low retention of CLT concepts from the Math 300 course. Lastly, we note that the median increase in the number of questions answered correctly from pretest to posttest as measured by the paired differences was 10.5 out of 27 (38.89%) for our Math 300 and 6 (22.22%) for our Math 400 students.
Because the Math 400 students had an extra semester of statistics, we were curious if there were any noticeable differences between their posttest performance and the posttest performance of the Math 300 students. Given the teaching and assessment methods used in each class, it seemed logical to divide the assessment tool questions into two broad categories: those that were more graphical and those that were more fact recollection and computational. Questions 4 and 5 were the assessment items that were more graphical in nature and most directly related to the Sampling SIM program and the corresponding activity used in Math 300. They were also very similar or the same as the assessment items used when developing and assessing the Sampling SIM program to improve students’ statistical reasoning (delMas, et al. 1999a, b, 2002). In Question 5, the students were given the graph of an irregular population distribution and five possible choices for the histogram of the sampling distribution (assuming 500 samples each of a specified size n). Figure 1 below shows the population distribution and sampling distribution choices (Choices A through E). Question 5a asked students to select which graph represents a distribution of sample means for 500 samples of size n = 4 (Answer = C). Question 5e asked students to select which graph represents a distribution of sample means for 500 samples of size n = 25 (Answer = E). Question 4 was similar but involved a skewed distribution (Appendix B). Questions 4 and 5, each with 7 items, contained 14 of the possible 27 questions on the assessment tool. The 13 remaining questions consisted mostly of items that asked the students to recall facts about sampling distributions of means and/or to apply these facts to perform routine probability computations.
Figure 1. Population Distribution (Top Left) and Possible Sampling Distributions (A-E) for the Irregular Population Distribution (please see the Assessment Tool Question 5 in Appendix B).
Table 2 below shows how each class performed on the posttest with the 27 assessment questions divided into the following two categories:
We found that our Math 400 students actually did quite well on the Fact/Computational type questions answering an average of 11.3 (86.8%) correct with a very small standard deviation of 0.76 (2.81%). This was much better than our Math 300 students who averaged 8.9 (68.9%) correct with a standard deviation of 2.75 (10.19%). We believe that our results in Table 2 make sense in terms of what was covered and emphasized in each of these classes. Because the Math 400 students had more experience throughout the semester applying results about sampling distributions to solve problems, it should not be surprising that they performed well on questions that asked them to recall and apply those results. However, even though we demonstrated the Sampling SIM software and other graphical simulations in-class, our Math 400 students did not seem to be able to extend their knowledge to a more graphical realm.
Math 300 Post (N = 18) | Math 400 Post (N = 7) | |
---|---|---|
Graphical Questions 4 and 5 Average Number (Percent) Correct with Standard Deviation. (Max Number Correct = 14) |
9.6 (68.3%) s = 2.66 (9.9%) | 7.6 (54.1%) s = 2.93 (10.9%) |
Fact/Computational Average Number (Percent) Correct with Standard Deviation. (Max Number Correct = 13) |
8.9 (68.9%) s = 2.75 (10.2%) | 11.3 (86.8%) s = 0.76 (2.8%) |
Average Total Number (Percent) Correct with Standard Deviation. (Max Number Correct = 27) |
18.5 (68.5%) s = 4.26 (15.8%) | 18.9 (69.9%) s = 2.97 (11.0%) |
Figure 2. Correct Identification of Sampling Distributions for Math 300 and Math 400 Students.
Reasoning Category (delMas, et al. (1999a)) | Number of Students (N = 18) | Percent of Students (N = 18) |
Posttest Reasoning Pair Irregular Distribution (Question 5) (n = 4, n = 25) |
---|---|---|---|
Correct | 5 | 27.8% | (C, E) |
Good (Large-to-Small Normal) | 2 | 11.1% | (B, E) |
Large-to-Small Normal | 7 | 38.9% | (A, E) |
Large-to-Small Population | 1 | 5.6% | (A, B) |
Small-to-Large | 2 | 11.1% | (E, D) or (E, C) |
Other | 1 | 5.6% | (C, D) |
There are several interesting items to note from Table 3. First, of the 6 students (33.3%) who were able to choose the correct sampling distribution for n = 4, five were also able to choose the correct sampling distribution forn = 25. Also, all of our 14 students (77.8%) who chose the correct sampling distribution for n = 25 appeared in the better (top three) reasoning categories. Clearly our students were having a difficult time with choosing the sampling distribution for the small sample size (n = 4). The most common answer for the sampling distribution for n = 4 was graph A (please see Figure 1 above). In Appendix C we show the distribution of our Math 300 students’ answer pairs among the reasoning categories for both the pretest and posttest for both the irregular (Question 5) and the skewed (Question 4) population distributions. Our results are consistent with previous studies (delMas, et al. 2002) in: 1) showing improvement in students’ reasoning from pretest to posttest; 2) illustrating students’ difficulties in interpreting the skewed distribution; and 3) demonstrating students’ struggles with finding the correct sampling distribution for n = 4. Please see Appendix C for more details.
Of interest to us was the large number of our students (12 out of 18) who did not choose the correct sampling distribution for n = 4. Before we could conjecture about why this was happening, we needed to determine if our students were getting incorrect answers because of graphical misconceptions about shape and/or variability (such as confusing “variability” with “frequency”) or because of some misunderstanding of sampling distributions. Student graphical misconceptions of shape and variability have been studied and documented at the introductory statistics level by several authors (Lee and Meletiou-Mavrotheris 2003, Pfannkuch and Brown 1996).
Question 5:
(a) Which graph represents a distribution of sample means for 500 samples of size 4? (circle one) A B C D E Answer each of the following questions regarding the sampling distribution you chose for Question 5(a): (b) What do you expect for the shape of the sampling distribution? (check only one) _____ Shaped more like a NORMAL DISTRIBUTION. _____ Shaped more like the POPULATION. (c) Circle the word that comes closest to completing the following sentence: less I expect the sampling distribution to have the same more VARIABILITY than/as the POPULATION.
Because the shape and variability of the sampling distribution graphs for the irregular population were very clear to compare to the population distribution, we computed our consistent graphical reasoning measure based on this population only. A student was defined to demonstrate consistent graphical reasoning if the sampling distribution chosen was consistent with their stated expected variance and shape of the sampling distribution as compared to the population (even if their choice of sampling distribution was incorrect). We called this measure consistent because if the sampling distribution they chose was not the same (in terms of shape and variability) as what they said they expected, then there was some inconsistency in their answer. Please see Appendix C for details on how we computed the number of students that demonstrated consistent graphical reasoning.
In Table 4 below is a comparison of the correct sampling distribution chosen versus consistent graphical reasoning from pretest to posttest for the Math 300 students. We saw significant improvement in the Math 300 students from pretest to posttest for both their selection of the correct sampling distribution and their demonstration of consistent graphical reasoning. Also, while only 33.3% (6 students) correctly identified the sampling distribution for the irregular population with n = 4 on the posttest, 77.8% (14 students) were consistent in their actual choice for the sampling distribution and their stated expected shape and variance of the sampling distribution as compared to the population. Furthermore, all of the students who were correct were also consistent for n = 4.
Determining the Sampling Distribution form the Irregular Population for Math 300 Students |
Sample Size n = 4 |
Sample Size n = 25 | ||
Pre-Test | Post-Test | Pre-Test | Post-Test | |
% Selecting the Correct Sampling Distribution (Number of Students from N = 18) |
5.6% (1) | 33.3% (6) |
5.6% (1) | 77.8% (14) |
% with Consistent Graphical Reasoning (Number of Students from N = 18) |
16.7% (3) | 77.8% (14) |
11.1% (2) | 83.3% (15) |
In contrast, for the seven Math 400 students, we saw little or no improvement in consistent graphical reasoning from the beginning of the semester to the end of the semester with at most only three students demonstrating consistent graphical reasoning (for n = 25 on the posttest) and only one student demonstrating consistent graphical reasoning for the remaining items (n = 4 pre and post, and n = 25 pre). While this may be due to the small sample size, these results are depressingly consistent with the low percentage of correct identification of the sampling distribution for the Math 400 students as was previously shown in Figure 2.
We find the consistent graphical reasoning results interesting for several reasons. First, via our NSF grant, we were using activities and graphical devices such as applets throughout the semester in our Math 300 class. Thus we were very surprised at the low percent of our students displaying consistent graphical reasoning on the pretest. Recall that the pretest for this class was administered late in the semester (around the tenth week of class). Thus we expected these students to graphically understand shape and spread and hence be consistent, even if not correct, with their choice of sampling distribution given their stated expected shape and spread of the sampling distribution as compared to the population. However, upon further examination we realized that the Sampling Distributions activity was the only assignment we gave that actually had the students investigating shape and spread in a graphical setting, albeit in the context of learning about sampling distributions and CLT. In the Math 400 class we also demonstrated graphical concepts using simulations and applets (including a teacher demonstration of Sampling SIM when we reviewed the CLT). However, we assigned no student activities during class time or outside of class, and the homework assignments were essentially of a theoretical or computational nature. We conjectured that by assigning the Math 300 students to work through the Sampling Distribution activity using the Sampling SIM applet (instead of the teacher only demonstrating Sampling SIM in class, as was done in Math 400), we enabled the Math 300 students to develop better graphical comparison skills than our Math 400 students.
Based on our results we do not believe that on the posttest the majority of our Math 300 students were having major difficulties with consistent graphical reasoning (such as confusing frequency with variance). Rather it appears that our students had some misunderstandings about sampling distributions. Thus we decided to further examine the reasoning pair rankings in light of our consistent graphical reasoning measure.
Using Table 5, we made some observations and conjectures about our students’ understanding of sampling distributions and the CLT. First we observed that of our 9 (50%) students who said they expected the sampling distribution to have a shape more like the population, all had chosen a sampling distribution with this property and were thus consistent in terms of shape. Also, all of these students chose the correct sampling
Posttest Reasoning Pair Irregular Population Distribution (Question 5(a)) (n = 4, n = 25) |
Number (Percent) of Students (N = 18) | n = 4 Sampling Distribution
Shaped More Like: (Question 5(b)) | Variability of n = 4 Sampling
Distribution compared to Population: (Question 5(c)) | Number with Consistent Graphical Reasoning | ||
---|---|---|---|---|---|---|
Answer | Number of Students | Answer | Number of Students | |||
(C, E) | 5 (27.8%) | Normal | 5 | Less | 5 | 5 |
(B, E) | 2 (11.1%) | Pop. | 2 | Less | 2 | 2 |
(A, E) | 7 (38.9%) | Pop. | 7 | Same | 4 | 4 |
Less* | 3 | 0 | ||||
(A, B) | 1 (5.6%) | Normal* | 1 | More* | 1 | 0 |
(E, D) or (E, C) | 2 (11.1%) | Normal | 2 | Less | 2 | 2 |
(C, D) | 1 (5.6%) | Normal | 1 | Less | 1 | 1 |
Totals | 18 | Normal Pop. |
9 9 |
Less Same More | 13 4 1 |
14 |
distribution for n = 4. We suspected that many of our students were not recognizing how quickly the sampling distribution becomes unimodal as n increases. This was not surprising since students are used to thinking of the CLT as a limiting result that doesn’t really take effect until the “magic” sample size of 30. Next we observe that the majority of our students (13 out of 18) correctly stated that they would expect the sampling distribution to have less variability than the population. For the two students who chose E for the sampling distribution, it may have been because they were not able to graphically estimate the magnitude of the standard deviation. For the three students who answered “less” but who chose A for the sampling distribution, we were not sure if they did so because they were either not able to estimate the magnitude of the standard deviation or they may have been confusing variability with frequency (due to the difference in heights of the histogram bars versus the height of the population distribution). Lastly, we thought the four students who answered “the same” (and were thus consistent in their choice of A for the sampling distribution) may be confusing the limiting result about the shape of the sampling distribution (i.e. as n increases the shape becomes approximately normal, via the CLT) with the fixed (i.e. non-limiting) result about the variance of the sampling distribution, regardless of its shape (i.e. the variance of the sampling distribution is , via mathematical expectation). As with the shape, these students may be thinking the variability result does not “kick in” until the sample size is greater than 30. Because of the nature of classroom research, we note that these subgroups consist of very small numbers of students and there could be other explanations for what we observed. We believe it would be very interesting to extend this initial research to see if our observations and conjectures above held for other classroom researchers or larger studies. Furthermore, we would want to follow up with personal interviews of students who used inconsistent graphical reasoning skills to gain additional insights into where the disconnect in learning occurs.
These observations led us to consider whether our students really knew and understood the basic result about the variance of the sampling distribution of the sample mean. Question 9d of the assessment tool (Appendix B) gave a quick statistic on this concept: 15 (83%) of our students stated it was “true” that if the population standard deviation equals then the standard deviation of the sample means in a sampling distribution (using samples of size n) from that population is equal to . All of the students in the top four rows of the table answered “true” to question 9d (except one who did not answer the question). Thus, we believed that our students were able to validate this fact when it was presented to them yet they did not understand it well enough to extend their knowledge to the graphical realm.
Answer to Question 9d (std. dev. of Sampling Distribution is ) | Answer to Question 2 | TOTAL | ||
---|---|---|---|---|
a, b, or e | c (uses pop. std. dev.) | d (uses standard error) correct answer | ||
True correct answer | 0 | 8 | 7 | 15 |
False | 0 | 3 | 0 | 3 |
Total | 0 | 11 | 7 | 18 |
We looked at other comparisons of theoretical knowledge of the standard deviation of the sampling distribution versus computational ability using that knowledge. Questions 6 and 9d of the assessment tool were essentially fact recollection questions concerning the magnitude of the standard deviation of the sampling distribution () given the sample size n versus the magnitude of the standard deviation of the population (). Questions 2 and 3 required students to use their knowledge of sampling distribution variability to perform a probability computation using the sample mean. In general, we found our Math 300 students performed well on their recall of theoretical knowledge of the standard deviation of the sampling distribution (11 of 18 answered both 6 and 9d correctly while 17 answered at least one correctly) but were not able to apply that theoretical knowledge (9 out of 18 missed both Questions 2 and 3). We were not sure why our Math 300 students were not correctly applying their theoretical knowledge but we conjectured that it could be because they have not fully realized the concept that “averaging reduces variability” and also did not recognize the sample mean as a random variable with a different standard deviation than the population from which the sampling was done. We also recall from Section 4.2.4 above that our Math 300 students were having a difficult time graphically understanding this concept for small n. We believed this may be due to the coverage late in the semester of these concepts and thus a lack of practice with the concepts. On the other hand, when we examined these same questions on the posttest of our Math 400 students, we saw that the majority understood the theoretical results and were able to apply them (6 of 7 answered both 6 and 9d correctly, 5 out of 7 answered both Questions 2 and 3 correctly, and 4 out of 7 answered all four questions correctly). This is also seen by their high score with low standard deviation on the Fact/Computation component of the post assessment tool (see Table 2 above). We should also note here that from the pretest results at the beginning of the semester, we saw poor retention by the Math 400 students of concepts as only 1 of 7 students answered both 6 and 9d correctly while 6 students answered at least one of these correctly, but 4 students missed both Questions 2 and 3 with only 1 student getting all four questions correct. However, by the end of the semester, via repeated application of sampling distribution concepts, we saw that the Math 400 students seem to demonstrate more accurate knowledge and application of the theory.
The survey instruments provided information about what class topics the students found to be difficult, which class activities the students valued as contributing positively to their learning experience, and their self perceived understanding of class topics. In the surveys we asked students to rank their current understanding of 65 probability and statistics concepts, such as the Central Limit Theorem, using a Likert scale of 1 to 5 where 1 indicated low knowledge and 5 indicated high knowledge. In Appendix C in Table C.4 we show student responses to topics of relevance for this paper. In general, for topics covered in the Math 300 class, we saw noteworthy increases in student self-assessment of their knowledge of these topics. Topics for which students had an average increase of more than 2 points on the scale (maximum possible increase is 4) included their understanding of: how to find probabilities associated with continuous random variables; mathematical expectation and its properties; the normal distribution; and the Central Limit Theorem. Topics for which the average rating for their understanding was at least 4 points on the post-survey included: how to compute a sample mean versus how to compute the distribution (population) mean; how to compute a sample variance versus how to compute the distribution (population) variance; and the normal probability distribution.
The post-survey also included a section where they were asked a series of open-ended questions concerning their reactions to methods and technologies used to teach the class. The responses to these questions were analyzed for patterns in responses. From the post-survey, students mentioned the CLT activity (13 out of 21, 62%) and group activities and group/individual reports in general (17 out of 21, 81%) as “contributing positively” to their learning. They also believed that the technology used in the class (Minitab and computer simulation activities) helped them learn the material (13/21 or 62%); that methods used in presenting the activities and/or class demonstrations stimulated their interest in the material (13/21); and that the class stimulated their problem solving skills (17/21). As these results indicate, students’ responses to our use of class activities and simulations were generally positive.
The end-of-course interviews with an external evaluator included 18 questions concerning the methods used in the course. Students were asked for their reactions to specific instructional strategies, such as the use of simulations, technology, Minitab, specific activities, as well as balance of lecture versus active student participation in the class. Many respondents to these interviews (7 out of 11) reported that they had the most difficulty understanding topics associated with distributions such as understanding the differences between continuous and discrete distributions; understanding the cumulative distribution function; and understanding some of the standard distributions such as the binomial and exponential distributions. Several students mentioned random variables, including independent random variables, functions involving sums of independent random variables, and the CLT as among the most difficult topics for them to understand.
This was shown by both our Math 300 students in their low pretest consistent graphical reasoning abilities and our Math 400 students in both their pretest and posttest consistent graphical reasoning abilities, and especially on their posttest scores when grouped by category (Table 2). These results were in line with observations by previous authors that computer simulation in itself does not stimulate increased understanding of concepts and thus “instructional materials and activities (need to) be designed around computer simulations that emphasize the interplay among verbal, pictorial and conceptual representation.” (delMas, et al. 1999a paragraph 3.11. Also see delMas, et al. 1999a, paragraphs 2.6-8 and 3.10-11 and Mills 2002 for an overview of the literature).
While our Math 300 students were certainly aware of the basic results regarding sampling distributions, they were not as proficient as our Math 400 students in applying the results to solve computational problems (see Table 2 and Section 4.3). This made sense because the Math 400 students had much more exposure to problem solving using these statistical concepts while the Math 300 students were introduced to the concepts at the end of the semester and thus had only a cursory exposure to problem solving with the concepts.
We saw this when we did a close examination of our Math 300 students reasoning pairs and consistent graphical reasoning. This was not surprising since students were used to thinking of the CLT as a limiting result that doesn’t really take effect until the “magic” sample size of 30.
We saw this particularly with our Math 300 students regarding their graphical reasoning pairs and consistency for small sample sizes and particularly in their application of the Empirical Rule in Questions 2 and 3. However, via more practice with these concepts, our Math 400 students seemed to grasp these concepts better (in the non graphical realm) in terms of knowing and applying the theory.
Difficulty in understanding these concepts has been well documented for students in introductory level statistics classes. We must “not underestimate the difficulty students have in understanding basic concepts of probability” and not “overestimate how well (our) students understand basic concepts.” (Utts, Sommer, Acredolo, Maher and Matthews 2003, section 5.2). Although the above quotes referred to algebra-based introductory level statistics, we believed that they are also applicable to post-calculus probability and statistics students. Certainly in our experience and via our classroom research we have found this to be the case. Given that a large number of our students reported difficulty with distributions (7 out of 11 responses on the end-of-course interview), it was not surprising that they also had difficulty understanding sampling distributions and the CLT.
We found the action research model useful for assessing our students’ understanding of sampling distributions and the CLT. While we had some preliminary ideas about our students understanding of these topics, it was enlightening to formally explore their understanding through the assessment data. Because of our research and the work of others (i.e. Rossman and Chance 2002), we have also questioned the purpose of our Math 300/400 sequence. What skills do we expect our students to have upon completion of the entire sequence or upon completion of only Math 300? How different are the answers to these questions if we have an “applied” versus a “theoretical” approach to the course? How well does the Math 300/400 sequence prepare our majors to teach high-school statistics, apply statistics in the real world, and/or go to graduate school? What will constitute “best practices” in teaching for achieving our (new) goals in the Math 300/400 sequence? We believe this is an exciting time to be teaching probability and statistics at all levels. We are excited by the work of our colleagues who are addressing the above questions in their research and development of teaching materials and methods. We also encourage instructors to engage in classroom research to assess how well their teaching is meeting their goals.
(This activity was provided by and slightly modified with permission from Rossman, et al. (1999) and is an earlier version of an activity by Garfield, delMas, and Chance (2000) which can be found on-line at www.tc.umn.edu/~delma001/stat_tools/).
Concepts: Random samples from populations, parameters, statistics, sampling distributions, empirical sampling distributions, the Central Limit Theorem.
Prerequisites: The student should be familiar with random variables, distributions (probability and empirical probability), expected value, and statistics such as the sample mean and sample variance.
Recap: In our last class we used simulation (the software package Sampling Sim) to examine the sampling distribution for the sample mean statistic . First we saw that the sample mean is a random variable. Our investigation of the empirical probability distribution of by taking many samples of the same size, n, from the same population resulted in the following observations about the sampling distribution of :
Population Parameters: mean = , standard deviation =
;
Sample Statistics: mean = , standard deviation = s;
Observations about the sampling distribution of :
Our simulation results to compute the sampling distributions for the sample mean statistic illustrated the Central Limit Theorem. This theorem says the following about the sampling distribution of the sample mean :
In this take-home activity, you will run the Sampling Sim program to investigate the sampling distribution for and thus see the Central Limit Theorem in action. The first part of this activity takes you through how to use the Sampling Sim program and reviews some basic concepts along the way (parts (a)-(i)). Once you know how to use the simulation and understand what information it is providing, please hand in parts (j)-(q).
To download Sampling Sim, go to the website: www.gen.umn.edu/research/stat_tools/ then click on the Software button and download the proper compressed file for your machine (zip for windows machines). I will also have this software installed on the computer labs in Madison Hall and the MLC.
Scenario: Professor Lectures Overtime
Let X = amount of time a professor lectures after class should have ended. Suppose these times follow a Normal distribution with mean = 5 min and standard dev = 1.804 min.
To investigate the sampling distribution of these values, we will take many samples from this population and calculate the value for each sample. Open the program Sampling SIM by double clicking on its icon.
The main point here is that results vary from sample to sample. In particular, statistics such as and s change from sample to sample. You will now look at the distribution of these statistics.
From the Windows menu, select Sampling Distribution. Move this window to the right so you can see all three windows at once. You should see one green dot in this window (it will be small and on the x-axis). This is the value from the sample you generated in (i). In the Sampling Distribution window, click on “New Series” so it reads “Add More.” Click the Draw Samples button. A new sample appears in the Sample Window and a second green dot appears in the Sampling Distribution window for this new sample mean. Click the Draw Samples button until you have 10 sample means displayed in the Sampling Distribution window. Note: You can click the F button in the Samples Window to speed up the animation. Record the values displayed in the “Mean of Sample Means” box and in the “Standard Dev. of Sample Means” box. These values are empirical. Compare these to the theoretical values predicted by the Central Limit Theorem.
Mean of Sample Means ________________
Standard Dev. of Sample Means ______________
Be very clear you understand what these numbers represent. If not, ask your instructor!
Sample Size (n) | Population Mean | Empirical Mean of Sample Means | Theoretical Mean of Sample Means (via the CLT) | Population Standard Deviation | Empirical Standard Deviation of Sample Means | Theoretical Standard Deviation of Sample Means (via the CLT) |
---|---|---|---|---|---|---|
1 | ||||||
5 | ||||||
25 | ||||||
50 |
(This document is the same content as the “Sampling Distribution Posttest” from Garfield, J., delMas, R., and Chance, B. (2000), tools for Teaching and Assessing Statistical Inference, hosted at www.tc.umn.edu/~delma001/stat_tools/ and used with permission.)
1. In a geology course, students were asked to determine the weight of rock samples. One instructor asked her students to weigh a rock several times on the same scale. This rock is known to weigh exactly 1000 grams. However, the scale is not completely accurate and sometimes it is off in either direction by 25 grams or less. After a lot of practice, one student weighed the rock 20 times, then computed and recorded the average of the 20 weighings. After a lot of practice, a second student weighed the rock 5 times, then computed and recorded the average of the five weighings. How would you expect the average weight recorded by the first and second student to compare? (circle one) a. The student who weighed the rock 20 times would have a more accurate average. b. The student who weighed the rock 5 times would have a more accurate average. c. Both averages would be equally accurate. d. It is impossible to predict which average would be more accurate. 2. Weight is a measure that tends to be normally distributed. Suppose the mean weight of all women at a large university is 135 pounds, with a standard deviation of 12 pounds. If you were to randomly sample 9 women at the university, there would be a 68% chance that the sample mean weight would be between: (circle one) a. 119 and 151 pounds. b. 125 and 145 pounds. c. 123 and 147 pounds. d. 131 and 139 pounds. e. 133 and 137 pounds. 3. If you took a random sample of 36 university women from the population described in question 2 above, there would be a 68% chance that the sample mean weight would be between: (circle one) a. 119 and 151 pounds. b. 125 and 145 pounds. c. 123 and 147 pounds. d. 131 and 139 pounds. e. 133 and 137 pounds. 4. The distribution for a population of test scores is displayed below on the left. Each of the other five graphs labeled A to E represents possible distributions of sample means for random samples drawn from the population. Population Distribution 4a) Which graph represents a distribution of sample means for 500 samples of size 4? (circle one) A B C D E 4b) How confident are you that you chose the correct graph? (circle one of the values below). 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Answer each of the following questions regarding the sampling distribution you chose for question 4a. 4c) What do you expect for the shape of the sampling distribution? (check only one) Shaped more like a NORMAL DISTRIBUTION. Shaped more like the POPULATION. Circle the word between the two vertical lines that comes closest to completing the following sentence. | less | 4d)I expect the sampling distribution to have | the same | VARIABILITY than /as the POPULATION. | more | 4e) Which graph do you think represents a distribution of sample means for 500 samples of size 16? (circle one) A B C D E 4f) How confident are you that you chose the correct graph? (circle one of the values below). 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Answer each of the following questions regarding the sampling distribution you chose for question 4e. 4g) What do you expect for the shape of the sampling distribution? (check only one) Shaped more like a NORMAL DISTRIBUTION. Shaped more like the POPULATION. Circle the word between the two vertical lines that comes closest to completing each of the following sentences. | less | 4h)I expect the sampling distribution to have | the same | VARIABILITY than /as the POPULATION. | more | | less | 4i) I expect the sampling distribution | the same | VARIABILITY than / as the sampling distribution I chose for question 4e to have | more | I chose for question 4a. 5. The distribution for a third population of test scores is displayed below on the left. Each of the other five graphs labeled A to E represent possible distributions of sample means for random samples drawn from the population. Population Distribution 5a) Which graph represents a distribution of sample means for 500 samples of size 4? (circle one) A B C D E 5b) How confident are you that you chose the correct graph? (circle one of the values below). 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Answer each of the following questions regarding the sampling distribution you chose for question 5a. 5c) What do you expect for the shape of the sampling distribution? (check only one) Shaped more like a NORMAL DISTRIBUTION. Shaped more like the POPULATION. Circle the word between the two vertical lines that comes closest to completing the following sentence. | less | 5d) I expect the sampling distribution to have | the same | VARIABILITY than / as the POPULATION. | more | 5e) Which graph do you think represents a distribution of sample means for 500 samples of size 25? (circle one) A B C D E 5f) How confident are you that you chose the correct graph? (circle one of the values below). 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Answer each of the following questions regarding the sampling distribution you chose for question 5e. 5g) What do you expect for the shape of the sampling distribution? (check only one) Shaped more like a NORMAL DISTRIBUTION. Shaped more like the POPULATION. Circle the word between the two vertical lines that comes closest to completing each of the following sentences. | less | 5h) I expect the sampling distribution to have | the same | VARIABILITY than / as the POPULATION. | more | | less | 5i) I expect the sampling distribution | the same | VARIABILITY than / as the I chose for question 5e to have | more | distribution I chose for question 5a 6. The weights of packages of a certain type of cookie follow a normal distribution with mean of 16.2 oz. and standard deviation of 0.5 oz. Simple random samples of 16 packages each will be taken from this population. The sampling distribution of sample average weight () will have: (CIRLCE ONE) a. a standard deviation greater than 0.5 b. a standard deviation equal to 0.5 c. a standard deviation less than 0.5 d. It’s impossible to predict the value of the standard deviation. 7. The length of a certain species of frog follows a normal distribution. The mean length in the population of frogs is 7.4 centimeters with a population standard deviation of .66 centimeters. Simple random samples of 9 frogs each will be taken from this population. The sampling distribution of sample average lengths (the average ) will have a mean that is: (CIRLCE ONE) a. less than 7.4 b. equal to 7.4 c. more than 7.4 d. It’s impossible to predict the value of the mean. 8. Scores on a particular college entrance exam are NOT normally distributed. The distribution of test scores is very skewed toward lower values with a mean of 20 and a standard deviation of 3.5. A research team plans to take simple random samples of 50 students from different high schools across the United States. The sampling distribution of average test scores (the average ) will have a shape that is: (CIRLCE ONE) a. very skewed toward lower values. b. skewed toward lower values, but not as much as the population. c. shaped very much like a normal distribution. d. It’s impossible to predict the shape of the sampling distribution. 9. Consider any possible population of values and all of the samples of a specific size (n) that can be taken from that population. Below are four statements about the sampling distribution of sample means. For each statement, indicate whether it is TRUE or FALSE. a. If the population mean equals , the average of the sample means TRUE FALSE in a sampling distribution will also equal . b. As we increase the sample size of each sample, the distribution TRUE FALSE of sample means becomes more like the population. c. As we increase the sample size of each sample, the distribution TRUE FALSE of sample means becomes more like a normal distribution. d. If the population standard deviation equals , the standard deviation TRUE FALSE of the sample means in a sampling distribution is equal to . The distribution for a population of measurements is presented below. Suppose that ten values are going to be sampled from this population and the sample mean calculated. Some possible values for this sample mean are 1, 6, 8, and 10.10. Which of the four possible sample mean values is MOST likely to be calculated? (circle only one) a. 1 b. 6 c. 8 d. 10 11. Which of the four possible sample mean values is LEAST likely to be calculated? (circle only one) a. 1 b. 6 c. 8 d. 10 12. Looking at the graph above, what would you guess to be the value of , the population mean? a. 0.0 e. 2.0 i. 4.0 m. 6.0 q. 8.0 u. 10.0 b. 0.5 f. 2.5 j. 4.5 n. 6.5 r. 8.5 v. 10.5 c. 1.0 g. 3.0 k. 5.0 o. 7.0 s. 9.0 w. 11.0 d. 1.5 h. 3.5 l. 5.5 p. 7.5 t. 9.5 x. 11.5 *This tool was provided to the authors by Robert delMas.
Figure C.1. Examples of pairs of graphs for each type of reasoning for Question 5 (Figure 9 of delMas, et. al. (2002), used with permission).
Table C.1 and Table C.2 below show how the answer pairs for Questions 4 and 5 were classified into reasoning pairs. The rows are the possible answers for the sampling distribution chosen for the smaller sample size and the columns are the sampling distribution chosen for the larger sample size. In parentheses by each answer, we state the characteristics of the sampling distribution, in terms of its shape and standard deviation as compared to the population.
Answer Pair Classification (4a, 4e) | 4e Answer (n = 16) | ||||
---|---|---|---|---|---|
4a Answer (n = 4) | A (Population/Same) | B (Population/Smaller) | C (Normal/Smallest) | D (Approx Normal/Smaller) | E (Approx Pop/Same) |
A (Population/Same) | Same | L-S Pop | L-S Normal | L-S Normal | S-L |
B (Population/Smaller) | S-L | Same | Good | Good | S-L |
C (Normal/Smallest) | S-L | S-L | Same | S-L | S-L |
D (Approx Normal/Smaller) | S-L | S-L | Correct | Same | S-L |
E (Approx Pop/Same) | L-S Pop | L-S Pop | L-S Normal | L-S Normal | Same |
Answer Pair Classification (5a, 5e) | 5e Answer (n = 25) | ||||
---|---|---|---|---|---|
5a Answer (n = 4) | A (Population/Same) | B (Population/Smallest) | C (Approx Normal/Smaller) | D (Population/Smaller) | E (Normal/Smallest) |
A (Population/Same) | Same | L-S Pop | L-S Normal | L-S Pop | L-S Normal |
B (Population/Smallest) | S-L | Same | S-L | S-L | L-S Normal |
C (Approx Normal/Smaller) | S-L | L-S Pop | Same | Other | Correct |
D (Population/Smaller) | S-L | L-S Pop | Good | Same | Good |
E (Normal/Smallest) | Other | Other | S-L | S-L | Same |
Below in Figure C.2 are graphs showing the percent of Math 300 students in each reasoning category (both pretest and posttest results) for both the skewed and the irregular populations (Questions 4 and 5, respectively). The figure also shows the distribution of the twenty-five possible answer pairs for each question (assuming equally likely answer pairs). Note that none of the twenty-five possible reasoning pairs for the skewed distribution where classified as other. The percent of students who were classified as having correct or good reasoning increased from 16.7% to 33.3% for the skewed population and from 11.1% to 27.8% for the irregular population. If we examine the students who were classified as correct, good, or large to small normal, then we see an increase from 27.8% to 61.1% for the skewed distribution and 11.1% to 77.8% for the irregular distribution. We note that our students seemed to have a more difficult time with the skewed distribution than the irregular distribution. In addition, we see a large decrease in the percent of students that show incorrect reasoning (small-to-large, same, or other) from pretest to posttest (55.6% to 22.2% for the skewed population and 77.8% to 16.7% for the irregular population). These trends and difficulties are consistent with previous results (delMas, et al., 2002).
Of interest to us was the large percent of our students that were classified as having “large-to-small normal” reasoning for the irregular population. Since 78% of our students chose the correct sampling distribution for the large sample size for the irregular distribution (see Section 4.2.1 above), it appears that our students were not demonstrating correct or good reasoning because of their choice of the sampling distribution for the small sample size.
Figure C.2. Distribution of Math 300 Students (and Equally Likely Reasoning Pairs) into Reasoning Categories for the Skewed and Irregular Populations (Questions 4 and 5, respectively)
Stated Variance of Sampling Distribution | Stated Shape of Sampling Distribution | 5a - Sampling Distribution Chosen for Irregular Population n = 4 (Variance of Sampling Distribution Compared to Population) | Totals | ||||
---|---|---|---|---|---|---|---|
A Population (Same) | B Population (Same) | C Normal (Smaller) Correct Answer |
D Population (Smaller) | E Normal (Smallest) | |||
Less than Population | Normal | (1) | 6* | 2* (2)* | 8 (3) | ||
Population | 3 | 2* (1)* |
* | 5 (1) | |||
Same as Population | Normal | (1) | (3) | 0 (4) | |||
Population | 4* | (1) |
4 (1) | ||||
More than Population | Normal | (1) | (3) | 0 (4) | |||
Population | 1 (1) | (3) |
(1) | 1 (5) | |||
Totals | 8 (2) | 2 (5) |
6 (1) | 0 (2) | 2 (8) | 18 (18) |
Cells with an asterisk indicate that the student displayed “consistent graphical reasoning” even if their answer was not correct.
Stated Variance of Sampling Distribution | Stated Shape of Sampling Distribution | 5e - Sampling Distribution Chosen for Irregular Population n = 25 (Variance of Sampling Distribution Compared to Population) | Totals | ||||
---|---|---|---|---|---|---|---|
A Population (Same) | B Population (Same) | C Normal (Smaller) | D Population (Smaller) | E Normal (Smallest)) Correct Answer | |||
Less than Population | Normal | (1) |
1* | 2 (1) | 14* | 17 (2) | |
Population | (1) | * |
* | 0 (1) | |||
Same as Population | Normal | (2) | (1) | 0 (3) | |||
Population | * (2)* | 0 (2) | |||||
More than Population | Normal | (2) | (1) | (1) | 0 (4) | ||
Population | (1) | 1 (1) |
(4) | 1 (6) | |||
Totals | 0 (8) | 1 (2) |
1 (1) | 2 (6) | 14 (1) | 18 (18) |
Cells with an asterisk indicate that the student displayed “consistent graphical reasoning” even if their answer was not correct.
Topic (Sample Size of 22 Students) | Post Avg. | Avg. Diff. (Post - Pre) |
Paired Std. Dev. |
---|---|---|---|
My understanding of how to compute a sample mean versus how to compute the distribution (population) mean. | 4.3 | 1.7 | 1.2 |
My understanding of how to compute a sample variance versus how to compute the distribution (population) variance. | 4.0 | 1.6 | 1.6 |
My understanding of empirical probability distributions versus probability distributions. | 3.9 | 1.8 | 1.3 |
My understanding of how to find probabilities associated with discrete random variables. | 3.7 | 1.9 | 1.2 |
My understanding of how to find probabilities associated with continuous random variables. | 3.8 | 2.0 | 1.2 |
My understanding of Bernoulli trials and their associated distributions. | 3.4 | 1.6 | 1.6 |
My understanding of the Normal Probability Distribution. | 4.1 | 2.1 | 1.3 |
My understanding of the difference between cumulative probability functions and probability mass/density functions. | 3.0 | 1.5 | 1.6 |
My understanding of Mathematical Expectation and its properties. | 3.5 | 2.1 | 1.2 |
My understanding of independent random variables. | 3.9 | 1.8 | 1.1 |
My overall understanding of random variables. | 3.6 | 1.4 | 1.2 |
My understanding of the Central Limit Theorem. | 3.8 | 2.2 | 1.5 |
My understanding of Point Estimation. | 3.1 | 1.3 | 1.6 |
My understanding of Confidence Intervals for Proportions. | 3.2 | 1.4 | 1.4 |
My overall understanding of probability. | 3.4 | 1.2 | 1.0 |
My overall understanding of statistics. | 3.0 | 1.0 | 1.2 |
Question 4a | Posttest | Totals | |
---|---|---|---|
Pretest | Correct ID | Incorrect ID | |
Correct ID | 0 | 0 | 0 |
Incorrect ID | 3 | 15 | 15 |
Totals | 3 | 15 | 18 |
Question 4e | Posttest | Totals | |
---|---|---|---|
Pretest | Correct ID | Incorrect ID | |
Correct ID | 1 | 2 | 3 |
Incorrect ID | 9 | 6 | 15 |
Totals | 10 | 8 | 18 |
Question 5a | Posttest | Totals | |
---|---|---|---|
Pretest | Correct ID | Incorrect ID | |
Correct ID | 0 | 1 | 1 |
Incorrect ID | 6 | 11 | 17 |
Totals | 6 | 12 | 18 |
Question 5e | Posttest | Totals | |
---|---|---|---|
Pretest | Correct ID | Incorrect ID | |
Correct ID | 0 | 1 | 1 |
Incorrect ID | 14 | 3 | 17 |
Totals | 14 | 4 | 18 |
Date: ____________________ Current Class/Semester: ___________________
Please list all probability and/or statistics courses that you have taken (other than this course) or write none if you have not had any previous courses of this type. If you have had some other exposure to probability and/or statistics please indicate that below.
Course Number & Name When & Where Taken Grade
Other Exposure to Probability and/or Statistics:
Please rate your knowledge of the given topic with 1 indicating low knowledge and 5 indicating high knowledge. It is important that you answer as honestly as possible. Some of these topics you may have not have been exposed to very much or at all.
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
In this E-mail Interview, I will ask some questions pertaining to the teaching strategies, materials, technology, and assignments used in this class. This is being done as part of the evaluation process for the National Science Foundation project in probability and statistics that Dr. Lunsford is co-directing. I am serving as the project evaluator and therefore I collect information pertaining to this course. Your responses will not be shared with Dr. Lunsford until after final grades have been posted. After grades have been posted, I will share a summary of your responses with Dr. Lunsford, however, individual students will not be identified. As Dr. Lunsford has shared with you, participating in this interview will result in bonus points, which will be added to your quiz scores. To aid you in answering the questions, I have included a table to refresh your memory concerning the topics covered in the class this term.
Thank you for providing us with your opinions. E-mail me if you have any questions.
Dr. Tracy Goodson-Espy
Table D-1: Major Mathematical Topics Covered
Topic |
---|
Sample Spaces, Outcomes, and Events |
Relative Frequency Histograms |
Basic Descriptive Statistics (Mean, Sample Mean, Variance, Sample Variance, etc. |
The Probability Function and its Basic Properties |
Methods of Enumeration (Combinatorics) |
Conditional Probability and Independent Events |
Bayes’ Theorem |
Basic Concepts for Discrete and Continuous Random Variables |
Mathematical Expectation |
Moment-Generating Functions |
Bernoulli Trials and the Binomial Distribution |
The Poisson Distribution |
The Uniform Distribution (Discrete and Continuous) |
The Exponential Distribution |
The Normal Distribution |
Distributions of Two Random Variables |
Independent Random Variables |
Distributions of Sums of Independent Random Variables |
The Central Limit Theorem |
Confidence Intervals for Means |
Confidence Intervals for Proportions |
Sample Size |
delMas, R. (2002), Sampling SIM, version 5.4 [On line]. www.gen.umn.edu/research/stat_tools/
delMas, R., Garfield, J., and Chance, B. (1999a), “A Model of Classroom Research in Action: Developing Simulation Activities to Improve Students’ Statistical Reasoning,” Journal of Statistics Education [Online], 7(3). jse.amstat.org/secure/v7n3/delmas.cfm
delMas, R., Garfield, J., and Chance, B. (1999b), “Exploring the Role of Computer Simulations in Developing Understanding of Sampling Distributions,” paper presented at the 1999 American Educational Research Association Annual Meeting, [Online]. www.tc.umn.edu/~delma001/stat_tools/
delMas, R., Garfield, J., and Chance, B. (1999c), “The Role of Assessment in Research on Teaching and Learning Statistics,” paper presented at the 1999 American Educational Research Association Annual Meeting, [Online]. www.tc.umn.edu/~delma001/stat_tools/
delMas, R., Garfield, J., and Chance, B. (2002), “Assessment as a Means of Instruction,” paper presented at the 2002 Joint Mathematics Meetings, [Online]. www.tc.umn.edu/~delma001/stat_tools/
Feldman, A. and Minstrell, J. (2000), “Action Research as a Research Methodology for the Study and Teaching of Science,” in Handbook of Research Design in Mathematics and Science Education, eds. Kelly, A. E., and Lesh, R. A., pp. 429 - 456, Lawrence Erlbaum Publishers, Mahwah, NJ.
Garfield, J. (2001), Evaluating the Impact of Educational Reform in Statistics: A Survey of Introductory Statistics Courses. Final report for National Science Foundation grant REC-9732404. Minneapolis, MN.
Garfield, J., delMas, R., and Chance, B. (2000), Tools for Teaching and Assessing Statistical Inference, [Online]. www.gen.umn.edu/research/stat_tools/
Garfield, J., delMas, R., and Chance, B. (2002), ARTIST: Assessment Resource Tools for Improving Statistical Thinking, [Online]. www.gen.umn.edu/artist/
Hogg, R.V., and Tanis, E.A. (2001), Probability and Statistical Inference, 6^{th} Edition, New Jersey: Prentice Hall.
Lee, C., and Meletiou-Mavrotheris, M., (2003), “Some Difficulties of Learning Histograms in Introductory Statistics,” in 2003 Proceedings of the American Statistical Association, Statistics Education Section [CD-ROM], pp. 2326 - 2333. Alexandria, VA: American Statistical Association.
Lunsford, M. L., Goodson-Espy, T. J., and Rowell, G. H. (2002), Collaborative Research: Adaptation and Implementation of Activity and Web-Based Materials into Post-Calculus Introductory Probability and Statistics Courses, funded by the Course, Curriculum, and Laboratory Improvement program of the National Science Foundation, awards DUE-0126600, DUE-0126716, DUE-0350724, DUE-126401, [Online]. www.mathspace.com/NSF_ProbStat/
Meletiou-Mavrotheris, M. and Lee, C. (2002), “Teaching Students the Stochastic Nature of Statistical Concepts in an Introductory Statistics Course,” Statistics Education Research Journal, 1(2), 22-37.
Mills, J. D. (2002), “Using Computer Simulation Methods to Teach Statistics: A Review of the Literature,” Journal of Statistics Education [Online] 10(1). jse.amstat.org/v10n1/mills.html
Noffke, S., and Stevenson, R. (eds.) (1995), Educational Action Research, New York: Teachers College Press.
Nolan, D. and Speed, T. (2000), Stat Labs: Mathematical Statistics Through Application, New York: Springer-Verlag.
Parsons, R. and Brown, K. (2002),
Pfannkuch, M. and Brown, C. (1996), “Building On and Challenging Students’ Intuitions about Probability: Can We Improve Undergraduate Learning?” Journal of Statistics Education [Online], 4(1). jse.amstat.org/v4n1/pfannkuch.html
Rossman, A. and Chance, B. (2002), “A Data-Oriented, Active Learning, Post-Calculus Introduction To Statistical Concepts, Methods, And Theory,” paper presented at International Congress On Teaching Statistics 2002, retrieved July 9, 2005, at www.rossmanchance.com/iscat/ICOTSpaper02.htm
Rossman, A. and Chance, B. (2006), Investigating Statistical Concepts, Applications, and Methods, New York: Thomson Brooks/Cole.
Rossman, A., Chance, B., and Ballman, K. (1999), A Data-Oriented, Active Learning, Post-Calculus Introduction to Statistical Concepts, Methods, and Theory (SCMT), funded by the Course, Curriculum, and Laboratory Improvement program of the National Science Foundation, award #DUE-9950476, [Online]. www.rossmanchance.com/iscat/
Rowell, G., Lunsford L., and Goodson-Espy, T. (2003), “An Application of the Action Research Model,” in 2003 Proceedings of the American Statistical Association, Statistics Education Section [CD-ROM], 3568-3571, Alexandria, VA: American Statistical Association.
Schau, C. (1999), Survey of Attitudes Towards Statistics, [Online]. www.unm.edu/~cschau/viewsats.htm
Schau, C. (2003, August), Students' Attitudes: The "Other" Important Outcome in Statistics Education, Joint Statistical Meetings, San Francisco, CA.
Siegrist, K. (1997), Virtual Laboratories in Probability and Statistics, funded by the Course, Curriculum, and Laboratory Improvement program of the National Science Foundation, award DUE-9652870, [Online]. www.math.uah.edu/stat
Terrell, G. (1999), Mathematical Statistics: A Unified Introduction, New York: Springer-Verlag.
Tobin, K. (2000), “Interpretive Research in Science Education,” in Handbook of Research Design in Mathematics and Science Education, eds. Kelly, A. E., and Lesh, R. A., pp. 487-512, Lawrence Erlbaum Publishers, Mahwah, NJ.
Utts, J., Sommer, B., Acredolo, C., Maher, M., and Matthews, H. (2003), “A Study Comparing Traditional and Hybrid Internet-Based Instruction in Introductory Statistics Classes,” Journal of Statistics Education [Online] 11(3). jse.amstat.org/v11n3/utts.html
M. Leigh Lunsford
Department of Mathematics and Computer Science
Longwood University
Farmville, VA 23909
U.S.A.
lunsford@longwood.edu
Ginger Holmes Rowell
Department of Mathematical Sciences
Middle Tennessee State University
Murfreesboro, TN 37132
U.S.A.
rowell@mtsu.edu
Tracy Goodson-Espy
Department of Curriculum and Instruction
Appalachian State University
Boone, NC 28608
U.S.A.
goodsonespy@appstate.edu
Volume 14 (2006) | Archive | Index | Data Archive | Information Service | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications