The Bill & Melinda Gates Foundation performed a study to answer the question: “Can well-crafted student surveys play an important role in evaluating teacher effectiveness?” What researchers discovered is incredibly important not only to students who rate their teachers, but also to parents who want the best educational experience for their child.
Let’s look at this and other studies correlating surveys to teacher effectiveness and quality.
Harvard University, Ronald Ferguson, December 2018
This study found that certain classroom characteristics were linked to teacher quality, and these characteristics could successfully predict student achievement. The scenario presented by the study was a classroom full of students who appeared at a glance to be well-behaved and successfully learning, but in reality were bored because the teacher valued memorization more than understanding. The teacher was actually indifferent to students’ feelings and opinions.
While end-of-year tests scored high enough to keep this teacher in place, there was no love of learning among the students. The reason administration and the teacher didn’t know this? No one asked the students.
Colleges and universities evaluate teaching. These surveys help administrators determine pay increases and promotion opportunities. And students can rate instructors in online surveys that other students use to select which classes/instructors they prefer. Students in elementary and primary schools don’t have this option.
Inside Higher Ed
On the flip side, studies show that student evaluations of teaching are deeply flawed. They bear little relation to teaching effectiveness because they’re biased and unreliable. The study found that students’ biases affect ratings rather than an objective view of a teacher’s capabilities. In essence, a student’s learning experiences are swayed by gender, class size, type of course being taught, and other irrelevant criteria.
Students perceive instructors differently based on characteristics like age, gender, race, and more. As a result, these perceptions could negatively affect outcomes for minority groups while punishing teachers who represent unpopular topics or courses.
Correlating student ratings with teaching quality
Studies show that students evaluating teachers (SET) are notoriously unreliable and biased. They should not be relied upon to make staffing decisions or tenure options. Unreliable student surveys disadvantage faculty, and punishes those who teach less “popular” courses.
As part of the above study, researchers wanted to answer the question, “If SETs are valid, reliable, and unbiased, what next?” How do administrators interpret SET scores without a level of bias and a fair basis to judge teachers’ performance? The study reports that even under “ideal” circumstances, SETs “still yield an ‘unacceptably high error rate.'”
The study goes on to say that the correlation between student ratings and teaching quality is “far, far stronger than even the most optimistic empirical research can support.” If this is the case, then SETs lead to correct decisions. In essence, this means the teachers with the highest evaluations are most likely the poorest teachers and those with the lowest scores are “better than the typical instructor.”
Take teacher evaluations with a grain of salt
Educational administrators should take student ratings of quality of instruction with a grain of salt. Studies show these surveys are biased and unreliable. However, administrations still use these evaluations to determine staffing and more because they’re both easy and inexpensive to capture and evaluate compared with other teaching assessments for quality.
The evidence cautions evaluating the results. Studies are split between an active computation of teachers and technology-based simulations of teacher scores. When studies simulate scores and compare job candidates across a field, they found faculty who were underperforming compared to their peers. What they found was that SET scores don’t correlate with quality or student learning. Thus, teacher surveys don’t always lead to a determination of teacher quality.
Even high scorers on the SET can be identified as “poor teachers.” The above research found that “19 percent of those with SET scores above the 95th percentile are no better than the median professor at teaching.”
Final thoughts
What does this mean for evaluating teachers’ quality? In essence, you can’t filter out idiosyncrasies in teacher surveys. What you can do is construct surveys that focus on independent evaluations, interviews with students, teacher observations, and peer reviews of teaching materials. This gives you a more accurate picture of a teacher’s proficiency than SET scores alone.
Surveying students, parents, teachers, and peers can give you a more accurate picture of teaching quality. Still, expect some bias and variations. To get a more accurate picture of teaching proficiency, use multiple forms of assessment to discover a teacher’s true performance.