It’s that time of the year again—assessment season. Actually, in academia, it’s always assessment season. And while many may be content to silently curse the heavens for their lot as instructors in the modern academy, I feel compelled to tempt the wrath of our rulers and lament aloud. My complaint can be summed up bluntly, if bitterly: Contemporary assessment is a tyrannical regime and a scourge of the college professor.
Granted, assessment means many things in many contexts. Ostensibly, it is benevolent and means something like “research into student learning and experience in relation to institutional and program goals.” In this sense, assessment, while it sometimes relies on first-order faculty evaluations of students in the form of course assignments, is a second-order evaluation of courses or programs to see if they are meeting their goals. The famous Principles of Good Practice published by the American Association for Higher Education in 1992 adds that “Assessment is not an end in itself but a vehicle for educational improvement.”
The desire to evaluate and improve student outcomes is surely praiseworthy. But I believe that assessment has evolved beyond its humble beginnings and is becoming a totalitarian and technocratic regime that seeks to observe and measure everything from students to instructors, courses, programs, events and everything in between. The result is that there is little that assessment doesn’t affect in higher education, to the detriment of everyone involved.
My critique proceeds by identifying three problems that I see with the reigning regime. First, much assessment is too noisy to be useful and is a distraction from the self-evaluation good teachers already do. Second, assessment distorts educational goals by restricting course objectives to what is measurable. Third, too much focus on assessment creates an environment that hampers student curiosity. However, while I am attempting to assess assessment, I want to be very clear that I do not take myself to be offering a comprehensive evaluation of it. I only want to articulate, on the basis of my own limited experience as a faculty member, something of why many instructors find assessment so infuriating.
Assessments Are Noisy and a Distraction
A number of years ago, at a previous institution, I was asked to be part of an assessment of philosophy courses that contributed to the university’s general education curriculum. The assessment involved using grades from assignments already employed in the context of the course to determine if our students were accomplishing the relevant general education objectives and, presumably, to prompt interventions to improve the courses if students’ work was not up to snuff.
As an instructor, this struck me as odd. If we treat students’ grades as data, that data is unavoidably ambiguous. That is, students’ grades are a mix of a host of different things that it’s hard if not impossible to tease apart. Among them: (a) a student’s interests and intellectual abilities, (b) student effort, (c) the instructor’s abilities as a teacher, (d) the fairness and rigor of the instructor’s grading, (e) the fairness of the tools (both the question in the assignment and the rubric) used for grading, (f) the operationalizability of the general education objective, and many more.
For example, if the department has multiple sections of a course and in some of those sections students do well and in others they don’t, there are too many variables to reliably tell what the problem is so that it can be addressed: Are some instructors better teachers or just easier graders? Perhaps one is just better at applying rubrics than another. If students in classes that run MWF in the afternoon do worse than students in courses that run TR in the morning, is that because the TR instructor is likely to draw more humanities majors than the MWF instructor due to students’ schedules or that students are more tired in the afternoon than in the morning? Perhaps it’s due to TR morning classes being populated by more upperclassmen than underclassmen. Or maybe it’s that students who are in less desirable time slots (like MWF afternoon) are there because, for whatever reason, they have lower executive function and that lower executive function is the cause both of them being in the class with an undesirable time slot and doing less well.
Over time, perhaps one could have a large enough sample size to do meaningful analysis, but the student body is constantly changing and so are instructors. In other words, the data that is extracted this way is bound to be so noisy as to be practically useless. While assessment is supposed to improve student outcomes, with so much noise in the data, it’s hard if not impossible to know exactly where an intervention—if needed—should be.
Moreover, the top-down imposition of norms of assessment is likely to distract from the kinds of fruitful and continuous troubleshooting that good instructors already engage in anyway. Any thoughtful instructor has had the experience of a lesson not going well or of realizing (perhaps while grading a set of papers) that the question that looked so good two weeks ago is actually confusing. Conscientious instructors are constantly in the process of assessing and redesigning their classes already. Unfortunately, the one thing that top-down assessment is likely to do is give those who are part of the assessment regime the illusion of understanding what is happening in instructors’ classes and therefore of being able to provide top-down technocratic fixes. Combined with the understandable employment incentive of assessors to find work that needs to be done, instructors are likely to be caught on a perpetual treadmill of busywork that keeps them from focusing on teaching.
The Assessment Regime Distorts Educational Goals
In education circles, assessment often comes up in connection with backward design, the idea that when an instructor is designing a course, they should start with the desired learning outcomes, then create assessments that will enable them to verify that the outcomes have been met and then, finally, construct the course content and learning activities that will equip the students to meet the course objectives by successfully completing the assessments.
I’m sympathetic to this approach to course design. The philosopher in me always wants to figure out where we’re trying to get first and then work backward. I think this is true of almost everything, from physical training to a renovation project in my house to designing a general education curriculum for a whole institution. “What are we trying to accomplish?” is a question that’s never far from my lips.
However, the presence of the assessment regime distorts course construction in a way that certainly doesn’t encourage and usually doesn’t even allow this backward design process to take place. Although he uses slightly different language, the noted education professor Elliot Eisner pointed out in an essay entitled “Five Basic Orientations to the Curriculum” that an undue focus on assessment ultimately results in determining in advance what counts as a legitimate course objective and what does not: “Form sets the boundaries within which the substantive goals of education can be articulated.”
Backward design is supposed to prompt professors to simply ask themselves, “What do I want my students to learn?” and then work backward to questions of assessment. But assessment requires that course outcomes be operational. That is, they are supposed to be observable, measurable or quantifiable. In practice, this means that if I have an objective that doesn’t admit of being measured or quantified, then it isn’t a legitimate course objective because it can’t be assessed in the way that the regime requires. In other words, the requirements of the assessment regime mean that, in practice, professors can only ask themselves, “What do I want my students to learn (that I can measure)?” Since any educational goal that can’t be measured is excluded by the assessment regime, the second step—thinking about assessment—has become, in effect, the first step, because assessment determines in advance what is and what is not a worthy educational goal.
For instance, as a professor, I might want my students to cultivate important dispositions or virtues—for them to become more honest, courageous or diligent. Or, I might want students to cultivate certain affections—for them to develop a love of learning, or enjoy the life of the mind, or to have the pleasure of partaking in the philosophical way of life. Similarly, I may want them to appreciate the truth and not simply passively accept what others tell them. I am not interested in arguing here that virtues, dispositions and the rest are good objectives for courses. I am simply arguing that insofar as we lack reliable, operational constructs that can be measured in the context of a class (and while there is a lot of work being done in this area, it’s not at all clear that we have them in the case of complex character attributes like virtue), they can’t be used as course objectives.
Rather than infer that since cultivating virtue and the like are worthy educational goals, we must be thinking about education the wrong way, the inference is often that these aren’t legitimate goals of education, because we can’t reliably measure them. Thus, as Eisner warned, we have a case of the assessment tail wagging the education dog.
Assessment Hampers Student Curiosity
Nowadays, not only whole courses but each segment or module of a course is supposed to be prefaced with student learning outcomes (SLOs). The SLOs tell the students what they are supposed to be learning in the present module of the course and are what the instructor will be assessing. The intent is to guide students’ attention toward important things. Now, in some disciplines—perhaps most obviously the natural sciences, engineering or mathematics, where everything builds on what came before—this might make sense. But in the humanities and perhaps in the liberal arts curriculum more broadly, it’s not at all obvious that this is a good approach. There, I think that we should be encouraging and feeding students’ open-ended curiosity. It’s one thing for a professor to characterize a difficult text in broad strokes to facilitate students’ own encounter with it, and another thing entirely to tell them what they should be getting out of the encounter with the text before it happens.
In humanities and the liberal arts, if we must have objectives, they should be broad and admit of multiple paths toward achieving them. Here, if nowhere else, students should have the feeling that college education is not about a job or preparing for the market but is a place for curiosity and exploration.
When we preface everything students encounter with narrow SLOs, which can be followed up with corresponding questions to assess learning, there is a danger that we unnecessarily circumscribe student attention and curiosity. Rather than cultivating a feeling that college is an open environment where students are encouraged to pursue whatever piques their interest, students are encouraged to be task-oriented and stay focused on preset goals. In such an environment, students are likely to narrow their attention to hunt for what they know they will be evaluated on and leave the rest. As Marshall McLuhan famously put it, “the medium is the message.” In effect, this will result in the college equivalent of teaching to (and learning to) the test.
It’s no wonder that students aren’t excited about education and feel that it increasingly amounts to a series of hoops to jump through. I believe this to be symptomatic of what the assessment regime does to education, and it’s nothing short of disastrous. What we might gain in terms of efficiency and increased numbers of students getting good grades may come at the cost of open-ended curiosity—the fostering of which is a much more useful course objective. Or at least it would be if it wasn’t already forbidden by the assessment regime.
A Way Forward?
To be sure, the intent behind the assessment regime is laudable. From accreditors down to instructors, assessment is meant to ensure that expectations are clear, that students are learning and that educational quality standards are upheld. But there are other ways of achieving these goals without paying such terrible costs. I propose that any use of data in assessment should be outweighed by expert judgment. And the resident experts in the relevant student learning are faculty. Consequently, a good first move would be to re-empower faculty.
Contrary to the popular caricature, and aside from a few rare exceptions, professors are not kings and queens of their own fiefdoms, lording it over their students. Most professors are contingent workers and don’t even have to get fired to lose their jobs—they can just “not be renewed,” as they say. Combined with a consumer-minded business model, this has resulted in a shift in power in the classroom away from faculty and toward students. The assessment regime itself has compounded this problem as professors are micromanaged by increasing layers of bureaucracy that compromise faculty’s ability to exercise their professional judgment about their own courses and classrooms—including course objectives, how to evaluate student knowledge and classroom policies.
If we want to ensure high standards of education, the best thing to do isn’t to micromanage what professors do in their courses—it is to hire more full-time faculty, decrease busywork, offer positive incentives for reflecting on their courses and give them better job security. This would enable faculty to do the kind of on-the-ground evaluation and improvement that conscientious teachers want to do anyway and exercise their expert judgment without being worried about upsetting a student or administrator.
In short, if we want good teaching and to ensure that standards are kept high, we can’t rely solely on data: We need the expertise of our faculty. The overthrow of the tyranny of assessment should coincide with a restoration of the ancien régime.
