Can You Explain That in Plain English?
Making Statistics Group Projects Work in a Multicultural Setting

Table 1

English	Spanish	Japanese	Swedish	Flemish
Mean	Media	平均	Medelvärde	Gemiddelde
Standard Error	Error típico	標準誤差	Standardfel	Standaardfout
Median	Mediana	中央値 （メジアン）	Medianvärde	Mediaan
Mode	Moda	最頻値 （モード）	Typvärde	Modus
Standard Deviation	Desviación estándar	標準偏差	Standardavvikelse	Standaarddeviatie
Sample Variance	Varianza de la muestra	分散	Varians	Steekproefvariantie
Kurtosis	Curtosis	尖度	Toppighet	Kurtosis
Skewness	Coeficiente de asimetría	歪度	Snedhet	Scheefheid
Range	Rango	範囲	Variationsvidd	Bereik
Minimum	Mínimo	最小	Minimum	Minimum
Maximum	Máximo	最大	Maximum	Maximum
Sum	Suma	合計	Summa	Som
Count	Cuenta	標本数	Antal	Aantal

6. Assessment

6.1 Goals and Framework of Assessment

Joan Garfield describes the current vision of assessment as "that of a dynamic process that continuously yields information about student progress towards the achievement of learning goals," (Garfield, 2000). She goes on to discuss the evolution of assessment from the assignment of grades to an integral part of the teaching and learning process, and in particular to say that the primary purpose should be to improve student learning.

Garfield recommends that an appropriate assessment framework include

• What to assess,
  
• Purpose of assessment,
  
• Who assesses,
  
• Method of assessment and
  
• Action/Feedback to be taken.
  

For the group projects, all aspects of this framework have evolved over the years.

Regarding "what to assess," when a project is assigned, all students receive a grade sheet rubric with several sets of criteria detailing what will be assessed and how value will be assigned for each criteria:

1. The collection, presentation and explanation of data;
   
2. The statistical content and validity of results;
   
3. The written communication of results and appendix;
   
4. The oral presentation.

I’ve found that the more specific and clear the criteria are, the higher the quality of the projects. For courses above the introductory level, less guidance may be appropriate in order to mimic a consulting experience, but for introductory courses with students coming from diverse backgrounds and cultures, I prefer this more structured approach. Please refer to Appendices A and B for examples of grade sheets/ rubrics I’ve used at the undergraduate and the graduate levels.

To make assessment a purposeful tool to enhance student learning, in addition to assigning a numerical value to each criterion, I include written comments for each providing clear indications of how to improve on future work. Comments praising what students have done well are also important to encourage good habits in the statistical investigation cycle. Additionally, as many students come with a negative view of their own ability to succeed in a quantitative, mathematical course, a little praise goes a long way in establishing self confidence. Students also have the option of submitting work at least three days early for review in order to benefit from feedback prior to submitting the final "graded" version.

Over the years, both the "who assesses" and the "method" of assessments have evolved. Originally I was the sole assessor of work using the grading sheet rubric and comments as described above. Currently, each project also includes elements of self and peer assessment.

6.2 Self Assessment

Garfield states that "engaging the student in self-assessment is a critical and early part of the assessment process, and that no major piece of work should be turned in without self-criticism," (Garfield, 2000). Having received detailed assessment criteria, the students have clear guidelines to use in assessing their own work. Since most groups tend to divide up parts of the projects, these guidelines help them to critically evaluate each other’s work during project preparation.

I stress on the first day that it is more difficult to produce original work than to critique another’s work but that almost any work can be improved by further discussion with colleagues, and I emphasize the need for the group members to complete all work well ahead of time in order that they may provide each other with helpful feedback. For example, once the data analysis stage is complete, one member may write the memo while another creates the PowerPoint presentation. Since all members earn the same grade for the final project, they recognize the benefit in reviewing each other’s work and making constructive suggestions. This group self-assessment process has double benefits, leading them to create work of higher quality and improving their critical reading skills.

On the day projects are due and presentations are made, each student individually fills out a grade sheet rubric for his own project. After I’ve graded the projects, I compare my evaluation to the students’ self evaluations and provide comments where our evaluations differ. I’ve found this process helps students develop more realistic expectations of their "grade." Doug Wittenberg presented a talk at JSM 2006 in which he added points to the students’ final exam if their self-predicted grades were within 3 points of his evaluation. I have not tried that in this context, but find it an interesting idea.

6.3 Peer Assessment

When I first began using peer critiques, it was as informal discussion after each oral presentation. One year a group of particularly talented and articulate "presenters" gave a presentation with stylish slides, convincing and engaging audience contact, and horrendously flawed statistical reasoning – concluding that the price of a barrel of Brent crude oil would remain between $26 and $30 for the next five years. When I asked the class for feedback, they were overwhelmingly positive. So impressed were they with the style, that not a single student had noticed the errors in content. From that day on, I have included explicit written peer assessment of content in the projects as a way to increase critical listening skills. The GAISE guidelines recommend that students should "know how to critique news stories and journal articles that include statistical information," and I try to extend this to statistical information presented in oral presentations.

For each oral presentation, students must evaluate their peers, writing at least two specific positive comments about what went well and two specific comments on how the project presentation could be improved. Two of the four comments must be content related and all must be constructive or regenerative. See Table 2 for examples.

Table 2: Examples of Peer Assessment Comments

What went well?

What can be improved?

Clear analysis of the correlation matrix Good idea to show all variables in a single chart with trend line Interesting analysis comparing male/female future work/study plans, but expected values less than 5 for chi-squared test Clear interpretation of slope for dummy variable fireplace and for lot size.

Scatter plots should be on same slide in order to compare better. Serious crimes should be in first column of correlation matrix for better visualization Could have done a hypothesis test to show that there is no statistical evidence that the two independent variables used in multiple regression are related R-sq almost 1 so it should be a strong line fit, group said it wasn’t P value of t-test for "male arrests" is bigger than 0.1 so they should not consider it in final model Clarify sampling method and population more

 

"While not easy to do in any culture, some cultures are more prepared to give and receive feedback than others, making the process of evaluating performance a potential cultural minefield," (Schneider & Barsoux, 2003, p. 239). At the Joint Statistical Meetings 2006, Huizhen Guo presented a moving and frank talk on Balancing Cultural Differences in Teaching Statistics, (Huizhen, 2006). She spoke of her difficulty in evaluating professors as an undergraduate as her culture taught that it was not the place of the student to evaluate professors. On the flip side, she addressed the difficulty she had upon receiving her own evaluations as a professor and the challenges she faces in accepting feedback from the students. In a study examining the relationships between bosses and subordinates (paralleled here by professor and student), Hofstede defines Power Distance as one of the dimensions of national cultures (Hofstede, 1997). The Power Distance Index indicates a relative measure of the level of "dependency" between bosses and subordinates in national cultures, with France, Spain, Asian and African nations showing a high index and the United States, Great Britain and Scandinavian nations showing a low index. The lower the index, the more comfortable subordinates feel disagreeing with bosses and the stronger the preference for a consultative style of decision making. These cultural issues therefore arise in much the same way in peer evaluations, so I must carefully approach the "minefield" of peer evaluations with each new group of students.

To break the ice and to provide practice, I often ask students to anonymously critique my class a few days before their first presentation session. "Exposing" myself to criticism in this way helps the students to accept the idea of critiquing each other. I distribute a summary of all comments in the following session and as a class we develop useful vocabulary for constructive criticism. Students work in pairs to identify comments that are useful, and to rewrite comments that are neither constructive nor regenerative. The entire discussion takes about 15-20 minutes of class time but has long-lasting implications on the quality of comments and the comfort level in critiquing.

For all projects, each group receives an anonymous summary of peer comments. These peer assessments have multiple benefits. Students are strongly motivated to receive positive feedback from their peers, they tend to react more positively to critiques from their peers than from professors, and by doing the critiques students improve their critical listening skills and develop the kind of evaluation skills that are transferable to many other academic and business situations. Additionally, students become actively engaged in all presentations rather than passive listeners. Project and critique quality improve dramatically throughout the two semesters as students gain confidence and a greater comfort level in giving and receiving constructive criticism. An added bonus is the more specific qualitative feedback I receive in student evaluations at the end of the semester which helps me to improve the course.

Each individual student also earns from -2 to +2 points for the quality of his/her written critiques. Each student’s final project grade is then calculated as the group grade adjusted by individual critique grade. The motivating effect of these extra points far exceeds their effect on the final grade.

7. Conclusion

Student response to project work has been overwhelmingly positive. Typical students’ comments on course evaluations are that they found math interesting and applicable for the first time, they improved their ability and their confidence in writing and presenting technical information and working in English, and gained valuable experience in working as a team on a common project. Often there are comments on the amount of work the course requires, however over the last 5 years more than 90% of numerical responses on a scale of 1 to 5 with 5 as "strongly agree" have been 4 or 5 to the questions "Do you find the quantity of projects appropriate" and "Do you find the quality of projects appropriate". Many students also now use some form of statistical reasoning and analysis in their final thesis projects before graduating, a promising sign of greater interest in and retention of material learned.

While I have only anecdotal data, since incorporating regular project work, I’ve received positive feedback from client disciplines suggesting students are better prepared for project work in other courses, are using statistics in other courses, and have strong critiquing skills.

One of the greatest challenges for the instructor, however, is that teaching with projects and especially incorporating peer feedback have increased the workload for these courses. For example, completing the peer feedback summaries adds to grading time and, to be useful, this must be done quickly after the presentation. One possible solution in larger classes is to photocopy critique sheets and then cut out the critiques, returning a stapled stack of comments to each group. Getting other colleagues on board and excited about these methods is another challenge. However, as Robert Wardrop argues in his article on using student projects (Wardrop, 2000), "on certain occasions a teacher should opt for a more difficult way to teach statistics." In my experience, the extra effort to teach this way is well rewarded by increased satisfaction from seeing students apply what they learn to new situations and develop skills they can use throughout their academic and professional lives. In addition, I have had the opportunity to learn about a variety of new topics based on my students’ choices of projects.

---

Appendix A: Grade Sheet for Undergraduate Regression Project

1. Data collection  (10%)

1.1 Select a topic of interest and collect data (either by using a resource already available or by collecting it yourselves). You must have at least 4 independent variables and one dependent variable.   (4%)

1.2. Clearly identify the population from which the sample is taken, the method of data collection, the source of the data, and comment on any limitations of the method/data.  (4%)

1.3. Comment on the dependent variable and its expected relationship with all independent variables. Show me your data BEFORE beginning the rest of the project.  (2%)

2. Simple linear regression  (25%)

2.1. Defend your choice of "best variable" to use in the simple regression model.  (3%)

2.2. Create an appropriate graph, write down the model, and interpret model coefficients.  (5%)

2.3. Evaluate the model in the context of your data – think in terms of statistical significance of relationship, strength of relationship, and explained variation.  (9%)

2.4. Make interval predictions using your model and comment these.  (5%)

2.5. Verify model assumptions.   (3%)

3. Multiple regression model:   (25%)

3.1. Discussed choice of variables and showed reasoning on what to (not to) include and why?  (5%)

3.2. Evidence of model building and consideration of changes to coefficients as variables added/dropped?  (4%)

3.3. Model coefficients interpreted?   (4%)

3.4. Model evaluated?  (5%)

3.5. Predictions made and commented?  (4%)

3.6. Model assumptions verified?  (3%)

4. Memo  (20%)

4.1. Introduction explaining research goals and source of data.  (3%)

4.2. Body of memo summarizing results.   (5%)

4.3. Clear conclusion about relationships found and how well the independent variables explain the dependent variable.  (2%)

4.4. Discussed any shortcomings of project, anything you would do differently next time, and possible follow up studies.  (2%)

4.5. Well written (grammar, punctuation, spelling, structure) and clear to the non-statistician.  (4%)

4.6. References to appendix? Appendix well lain out and clearly labeled?  (4%)

5. Oral Presentation  (20%)

5.1. Voice projection, audience contact, posture, body language, engaging?  (4%)

5.2. Slides: style, spelling, clarity, dynamic?   (4%)

5.3. Teamwork evident?   Presentation time split amongst members?  (4%)

5.4. Purpose, conclusions clear?   Choice of "best model" clear?  (8%)

---

Appendix B: MBA Grading Rubric for Descriptive Statistics Project

	0-6	7-8	9-10
Data collection	Data collection method inappropriate or only vaguely described.	Data collection method explained and appropriate.	Data collection method well explained, appropriate, with any limitations noted.
Data presentation	Several graphs poorly labeled, some graphs inappropriate for goals, graphs on the whole did not add much insight into the issues discussed. Numerical measures inappropriate or interpretation did not add to understanding of data.	Most graphs clearly labeled and appropriate for type of data and goals, graphs generally aided in understanding of arguments made in write-up. Numerical measures and their interpretations generally correct and add some insight into data.	All graphs clearly labeled and appropriate for goals, showed creativity in creating graphs to make arguments clear and graphs fully supported write-up. Numerical measures interpreted well and contributed to understanding of data set.
Business write-up	Often superficial or insufficient reference to context, described aspects of graphs rather than the information they displayed, Little effort to go beyond context and look at implications for business decisions. Appendix poorly organized or superficially commented.	Generally well written with reference to context; analysis correct, some effort to look at implications. Written at a level accessible to a non statistician. Appendix referred to in text and clearly lain out.	Clear and well written, continual reference to context; Sound analysis and went beyond current context to look at implications of information gathered on future business decisions. Written at a level accessible to a non statistician. Appendix referred to in text, clearly lain out and commented.
Oral presentation	Transitions between team members not smooth or unbalanced participation by members or poor time keeping. Slides plain or contain some grammatical or spelling errors, lack structure. Project goals and conclusions less than clear to the audience.	Some teamwork evident, transitions fairly smooth, and time limit respected. Generally positive audience contact with room for improvement on voice projection or posture. Slides correct and support presentation. Goals and conclusions of project clear to audience.	Teamwork evident with smooth transitions, good timing and shared participation by all members; Excellent audience contact, voice projection, posture; Clear, dynamic and correct slides that enhance presentation; Purpose, conclusions and limitations of project clear to audience.

---

Acknowledgements

I presented an earlier version of this paper under the same title at the Joint Statistical Meetings in Seattle, 2006 and it was also included in the proceedings. Parts of this paper on peer assessment appeared in the proceedings of the IASE Satellite Meeting on Assessment held in Guimares, Portugal, 2007. The title was "Using Peer Assessment of Project Presentations to Develop Skills as Consumers of Statistical Information."

---

References

Apigian, C., & Gambill, S. (Winter 2004-2005). Is Microsoft Excel 2003 Ready for the Statistics Classroom? Journal of Computer Information Systems, 27-35.

Chance, B. (2000). Experiences with Authentic Assessment Techniques in an Introductory Statistics Course. In T. e. Moore, MAA Notes #52 Teaching Statistics: Resources for Undergraduate Instructors (pp. 209-218). Washington DC: Mathematical Association of American and American Statistical Association.

Cryer, J. (2001). Problems with Using Microsoft Excel for Statistics. Proceedings of the 2001 Joint Statistical Meetings. Alexandria, VA: American Statistical Association.

Fillebrown, S. (1994). Using Projects in an Elementary Statistics Course for Non-Science Majors. Journal of Statistics Education , 2 (2), online.

Garfield, J. (2000). Beyond Testing and Grading: New Ways to Use Assessment to Improve Student Learning. In T. e. Moore, MAA Notes #52 Teaching Statistics (pp. 201-207). Mathematical Association of America and American Statistical Association.

Guidelines for Assessment and Instruction in Statistics Education.(n.d.). Retrieved August Monday, 2008, from jse.amstat.org/Education/gaise/GAISECollege.htm

Hofstede, G. (1997). Culture and Organization: Software of the Mind. New York: McGraw-Hill.

Huizhen, G. (2006). Balancing Cultural Differences in Teaching Statistics. Proceedings of the Joint Statistical Meetings. Alexandria, VA: American Statistical Association.

MacGillivray, H. (1998). Developing and Synthesizing Statistical Skills for Real Situations through Student Projects. ICOTS 5, Israel (pp. 1150-1156). International Association of Statistics Education.

Moore, T. e. (2000). Section 2: Teaching with Data. In ec., MAA Notes #52 Teaching Statistics (p. section 2). Mathematical Association of America and American Statistical Association.

Nash, J. (2006). Spreadsheets in Statistical Practice - Another Look. The American Statistician , 60 (3).

Roseth, Garfield, Ben-Zvi (2008) cited in 2. needs reference

Schneider, S., & Barsoux, J. (2003). Managing Across Cultures, 2nd ed. Pearson Education Limited.

Wardrop, R. (2000). Small Student Projects in an Introductory Statistics Course. In T. e. Moore, MAA Notes #52 Teaching Statistics (pp. 19-25). Mathematical Association of America.

Williamson, D., Mendel, M., Tarr, J., & Yoklic, D. (2005). Student Manual for Essential Mathematics for Business Decisions: Part 1: Probability and Simulation, 2nd ed. Mathematical Association of America.

---

Michelle Sisto
International University of Monaco
2 Avenue Prince Albert II
Monte Carlo 98000
Principality of Monaco
Phone: +377 93 150 610 (home)
Fax: +377 92 052 030 (University)
msisto@monaco.edu

---

Volume 17 (2009) | Archive | Index | Data Archive | Resources | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications