How One Professor Mines Student Comments to Improve Her Teaching
Lisa Elfring, assistant vice provost in the Office of Instruction and Assessment, uses word clouds to make sense of comments in students' course evaluations.
This word cloud shows the most commonly used words in evaluations from one of Elfring's courses.
This cloud shows the most commonly used negative words.
This article originally appeared in Teaching – a weekly newsletter from The Chronicle of Higher Education that covers teaching and learning – and was reprinted with permission. The newsletter is produced by Senior Editor Dan Berrett, Senior Writer Beth McMurtrie and Senior Reporter Beckie Supiano.
When Lisa Elfring taught introductory biology at the University of Arizona, she struggled to make sense of the feedback students gave in their course evaluations. There was simply too much of it. With some 500 students enrolled, Elfring received 1,500 lines of comments.
Working on contract as a lecturer at the time, Elfring wanted to find a way to visualize the feedback for her annual review.
Fast forward to a couple of years ago, when she took an administrative position as the assistant vice provost in the office of instruction and assessment – the office responsible for course evaluations. She approached another office on campus, analytics and institutional research, to ask if it would be possible to turn the feedback from evaluations into word clusters and word clouds. Her colleagues there quickly came up with a tool.
This kind of presentation can't tell the full story of what students thought of a course, Elfring said. It serves, rather, as a jumping-off point. "It gives me an ability," Elfring said, "to make hypotheses. And then I can dive back into the data and test them."
Here's an example. One of the words that jumped out to Elfring in her negative word cloud was "cheat." Were students, she wondered, cheating in the course? When she went back to the raw data – the actual comments students had made – it quickly became clear that students used the word "cheat" in reference to the "cheat sheet" she allowed them to use. Context matters.
The tool is a little clunky right now, Elfring said, because data must be downloaded from one system and uploaded into another. But the instruction-and-assessment office is working on a fix, and Elfring aims to provide visualizations of their feedback to all university instructors as a matter of course.
Rick Michod, a professor of ecology and evolutionary biology, has already tried it out. Michod had long read student course evaluations, he said, "but I didn't have any way of thinking about them." To Michod, the data visualizations provide a way to look at student comments more objectively, as data points.
Doing so can still require some courage. When he got his word clouds, Michod noticed a term in a small font – just four of his 150 or so students had used it. But the word still concerned him: "rude." That's not how he saw himself. Digging into the comments, Michod learned that the students who used it had described a couple of incidents that had felt "awkward" to him when they happened.
Michod had just abandoned lecturing in favor of active learning. As part of the switch, he called on a larger number of students than the usual suspects he had engaged in the past. When he did, he was taken aback by how unprepared some of them were.
In two instances, Michod said, he "stumbled" choosing his words after a student gave an uninformed answer. "Students really felt that I had embarrassed them," he said.
Since then, Michod has crafted language to use in similar situations. "That's interesting. Tell me how you got to that thought," he'll now say.
"No matter what the response is," Michod said, "a good teacher uses the moment to open the door."
Want to receive the Teaching newsletter? Subscribe to it here. To learn more about The Chronicle of Higher Education's coverage of teaching and learning, click here or check out this FAQ.