A new tool developed at Cummings School of Veterinary Medicine aims to save time for instructors and help them better deliver feedback to students
Laurence Sawyer, assistant teaching professor and director of the Luke and Lily Lerner Clinic at Cummings School of Veterinary Medicine, oversees third-year veterinary students performing spay/neuter surgeries. Photo: Alonso Nichols
When third-year veterinary students at Cummings School of Veterinary Medicine at Tufts University step into the operating room to undertake their inaugural surgery, the energy in the room is palpable: there’s excitement, but also nerves.
“It's actually kind of a fun energy to harness,” says Yuki Nakayama, D.V.M., V14, an assistant clinical professor at Cummings School of Veterinary Medicine. “But some students are just terrified. We try to identify them early so we can get them to a place where they're comfortable enough to learn.”
During the surgery—a canine spay procedure—Nakayama and Laurence Sawyer, D.V.M., V99, assistant teaching professor and director of the Luke and Lily Lerner Clinic, pivot from student to student, overseeing multiple, overlapping surgeries. In addition to managing student emotions, they juggle questions, check-ins, and feedback. All of this must happen in under three hours. The fast-pace and high stakes mean that some students “may be in a mindset where they can’t process all the information,” says Sawyer.
To ensure students come away with the proper learning outcomes, instructors can spend another hour with each student afterwards, writing up narrative feedback on their performance. Sometimes, there isn’t time, and instructors must try to recall those details later when their schedule allows.
It can be time-consuming and imprecise, so the instructors felt there must be a better way to deliver accurate and clear feedback to students. Nakayama and Sawyer, collaborating with Ariana Hinckley-Boltax, D.V.M., PGDipVetEd, assistant teaching professor at Cummings School of Veterinary Medicine, looked to technology now flooding medical offices: AI.
Doctors and veterinarians across the world are now using artificial intelligence as a digital scribe, to analyze diagnostic images, and to detect certain conditions.
“It’s coming whether we like it or not,” says Hinckley-Boltax.
Improving Feedback
The group thought the tech could address the challenges they faced in the classroom. So, they worked to build an AI-powered platform that captures and documents feedback provided in real-time and in less formal situations — when instructors provide guidance during a lab, for instance. The team expects the tool will enrich the learning process for their students while saving instructors’ time.
“Those day-to-day conversations are filled with feedback,” says Hinckley-Boltax. “A lot of those conversations are not captured in any sort of meaningful way.” The new application, VetFeedback.ai, fills that gap, acting as what Hinckley-Boltax calls a “transcription service on steroids.”
Instructors working in labs, sterile surgery, or clinics can record directly from a cellphone in their pocket, explains Sawyer, that will pick up the instruction and feedback they provide throughout the day. Then that audio is analyzed by VetFeedback.ai.
“Students always want more feedback. Whenever I talk to students—and even my previous experience as a student myself—you want as much feedback as possible to be the best possible doctor you can.”
The team has trained the platform on evidence-based feedback guidelines for veterinary care and veterinary education frameworks. In addition to transcribing audio captured during conversations, the platform applies what was said to those guidelines to show how it aligns with specific learning outcomes that the instructors can customize.
“We as instructors take a lot of effort to craft learning outcomes that are specific, measurable, and related to the most important aspects that we need students to accomplish,” says Hinckley-Boltax. But “it can get lost in these practical situations,” where there is a lot of cognitive load associated with accomplishing a task or learning a specific skill in the lab.
“There’s so much going on through a clinical day, a student can lose the forest for the trees in terms of what they're supposed to be taking away,” she adds.
Research from Hinckley-Boltax and colleagues at several other veterinary colleges around the U.S. recently found that students often do not recognize instructive feedback when it is delivered outside the scope of a formal assessment. In a survey that examined feedback offered during clinical rotations that was distributed to students and instructors at 33 universities, students reported receiving feedback only 28.5% of the time, though instructors said they had given it 44.5% of the time.
“Interpreting feedback and internalizing feedback is an emotional and subjective process,” says Hinckley-Boltax. “What I learned is that there really is a disconnect between students and instructors in how they view feedback.”
Yuki Nakayama (left), assistant clinical professor at Cummings School of Veterinary Medicine, works with a veterinary student on a procedure. VetFeedback.ai lets instructors working in labs, sterile surgery, or clinics record directly from a cellphone in their pocket that will pick up the instruction and feedback they provide. Photo: Alonso Nichols
Time Is Money
In addition to making feedback processes clearer for students, using AI as an instructional tool can save significant time and money, according to Hinckley-Boltax. Veterinary faculty, she explains, work from morning to night fulfilling clinical duties, designing lectures, answering emails, conducting research, and grading student work. That leaves little time to capture and then categorize feedback provided in off-hand conversations. But AI can pick up on those small interactions and place them in the context of their larger relevance to curriculum material.
Ultimately, the feedback is the instructor’s own, though. “We trained the AI to be minimally creative,” explains Hinckley-Boltax. “The AI is doing the legwork of summarizing and connecting to learning outcomes, but the instructor still reviews it to ensure that it's accurate.”
And the feedback isn’t just for students; if instructors use the platform often enough, the AI will begin analyzing their communication style to provide feedback on the feedback itself.
In April, the team presented the platform at the annual American Association of Veterinary Medical Colleges conference.
The tool is still in its early stages; Nakayama and Sawyer are just beginning to test its performance during clinical skills labs. But they hope to have a workable model in the fall.
“Students always want more feedback,” says Hinckley-Boltax. “Whenever I talk to students — and even my previous experience as a student myself — you want as much feedback as possible to be the best possible doctor you can.”
