September, 1992
The term "assessment" is often interpreted in different ways. Some people immediately think of tests and grades. Others think of evaluation. Before discussing assessment, I'd like to offer some definitions for assessment and related terms (Garfield and Corcoran 1986).
TEST is the narrowest of the terms, defined as the presentation of a standard set of questions to be answered.
MEASUREMENT often connotes a broader concept than test: we can measure characteristics in ways other than by giving tests (e.g., using observations, rating scales, etc.). Measurement can refer to both the score obtained and the process used.
EVALUATION is often defined as the determination of the congruence between performance and objectives, and connotes a professional judgment or the process that allows one to make such a judgment about the desirability or value of something.
ASSESSMENT has historically referred to the diagnosis of an individual's problems, but today is used more broadly as a synonym for evaluation. It is used to refer to the process of gathering data and assembling evidence so that judgments of value can be made. We often refer to "assessment of student learning," which means measuring what students have learned and assigning a judgment to that measurement.
In summarizing the group reports on assessment from the SHIFTY meeting, I noticed some general categories of discussion. They are: what to assess, how to assess, concerns related to assessment, how to use assessment information to improve instruction, and how ideas of TQM and assessment are related. I've begun the summary with an additional category: why assess. Throughout the summary, I've referred to material from the articles distributed at the meeting by Cross and Wiggins.
Many instructors believe that the purpose of assessment is to assign grades to students. I'd like to argue that the purpose of assessment should be to improve learning and teaching. Consequently, assessment can be viewed as a vehicle for gathering information about student learning that can be used to provide feedback to students about their learning as well as feedback to the instructor about students' achievement of course goals. Assessment offers us a way to reflect about what we're doing and what's really happening in our classes, as a type of "classroom research" (Cross). It offers a systematic way to gather and evaluate information to use to improve our knowledge of teaching statistics. Assessment may be used to identify what's not working, or where the "defects" are. It may also be used to help students become more aware of their own success in learning, and as a way for them to improve their self-assessment skills (Wiggins).
What we assess should be what we value. "Students should be assessed on real performances and use of knowledge" (Wiggins). But first, we need to determine what it is we really value: what we want students to know and be able to do as a result of taking a statistics course. This should be translated into clearly articulated goals and objectives (both broad and narrow), in order to determine what types of assessment are appropriate for assessing the attainment of these goals. Course goals and objectives as well as the criteria for evaluating them should also be made explicit to students (Wiggins). One way to begin thinking about goals for the course is to think about what students will need to be able to do in future courses or jobs.
In assessing the broad goals for a course, one form of assessment that is relevant to consider is "authentic assessment," or forms of students' work that reflect real-life situations and challenge students' ability to apply what they have a learned in those situations (Archbald and Newmann l988). In a statistics course, this might translate into using reports of students' analysis of a data set or solving a real statistical problem as a form of assessment.
In addition to evaluating the overall impact or effectiveness of an entire course, assessment may be used to evaluate different components or aspects of a course, such as student learning of a particular topic, computer labs, or the effectiveness of a particular activity.
There are many ways to gather assessment information, and several of these methods should be used to evaluate different aspects of a course.
For example:
Tabulating course attendance and/or lab participation
Obtaining feedback from other faculty who teach subsequent courses in a sequence
1 minute papers on a particular class
Questionnaires during or at the end of a course
Exams
Group projects
Quality improvement teams to provide a consistent flow of information
Portfolios of students' work
Tasks where students are required to apply ideas to different contexts
Tasks requiring students to evaluate or interpret information in an article or research report. For example, ask students to explain a paragraph from a research study which uses confidence intervals.
Rather than using a holistic method of assigning a grade to student projects, other alternatives are available. One method is to give a grade of A or "needs work," where students are allowed (and encouraged) to revise the paper or report until it meets the instructor's standards. Another way to evaluate group or individual projects or written papers is to use a scoring rubric. For example, points may be given in each of the following categories (using a 0, 1, 2 scale, for example):
A concern was raised about the time required to read and grade essays or student projects, when faculty are always so short of time. Another concern was that faculty are rewarded for research, and not for putting extra time into teaching-related activities.
Another concern regards being able to come up with a set of acceptable goals and objectives for courses. In some departments there is no clear focus on what should be, for example, the "statistics" course for all engineers at a particular institution.
Assessment of long-term effects of a course is desirable, but difficult to implement. How do we find and assess students, say six months or one year after completing a course? How do we find out how well they do in sequential courses?
Concerns related to assessing group work were also raised. For example, what is a fair way of scoring group projects when some students did not actively participate? My reaction to this is to restructure and monitor group activities to insure that this does not happen, and/or have students rate their own contributions to the project as well as each other's contributions.
TQM is a style of management that involves looking at and trying to continuously improve a process. Applied to education, the process can be viewed as what happens in the classroom, with the instructor viewed as the manager, who creates and manages the environment. Either the students themselves or the potential employers can be viewed as customers.
Ideas of TQM appear to be relevant to improving teaching and learning. The fundamental idea is that feedback from students and other clients can help provide a "road map" to improvement. Assessment feedback should be fast (e.g., minute papers) so that adjustments and improvements can be made in a timely way. The combination of TQM and assessment can be viewed as one way to build a partnership in learning between teachers and students.
TQM Methods which can be used in a course in ways such as these:
The connections between TQM, assessment, and Classroom Research (as described in the Cross article) are apparent when applying the PDCA (Plan, Do, Check, Act) steps of TQM to a course:
Plan: identify something in the process that needs improvement. (e.g., students have only used small, canned data sets in class, and don't have the skills needed to work with real data sets that may be large, messy, and have missing values, outliers, etc.)
Do: run an experiment where a change is made and the result is investigated. (e.g., introduce an activity in class where students have opportunities to analyze real data sets)
Check: collect data to determine if the desired change has been effective. (e.g., assess students to see if they show proficiency in analyzing real data sets)
Act: if it's successful, implement the change. If not, learn from the experiment and design another one. (e.g., introduce more activities in class involving real data sets or modify the way real data can be used in class activities or projects)
Archbald, D. A., and Newmann, F.M. (1988) Beyond Standardized Testing: Assessing Authentic Academic Achievement in the Secondary School. Reston, VA: National Association of Secondary School Principals.
Cross, P. and Angelo, T. (1988) Classroom Assessment Techniques: A Handbook for Faculty. National Center for Research to Improve Postsecondary Teaching and Learning, Ann Arbor, Michigan. Note: A new edition of this book will soon be available from Jossey-Bass Publishers. Their phone number is 415-433-1767.
Garfield, J. and Corcoran, M. (1986). Assessment in American Higher Education: An Historical Perspective. Background paper for the Conference on Assessment in Higher Education, University of Minnesota.
Stenmark, J. K. (Ed.) (1989) Assessment Alternatives in Mathematics: An Overview of Assessment Techniques That Promote Learning. Berkeley, CA: Lawrence Hall of Science Publications.
Stenmark, J. K. (Ed.) (1991) Mathematics Assessment: Myths, Models, Good Questions, and Practical Suggestions. Reston, VA: National Council of Teachers of Mathematics.
Note: Roy Erickson (MSU) feels that it's most important to obtain fast feedback in the presentation of his new course and has agreed to report his experiences to interested parties, including examples of questionnaires, how he used this information, and how he thinks it might have improved his course.