Optimizing AI Onboarding in Radiology: Impact of Training Case Number Using Multiple Sclerosis Lesions on 3D FLAIR, a pilot study
This pilot study involving thirteen radiology clinicians demonstrates that onboarding with three representative multiple sclerosis training cases on 3D FLAIR MRI significantly enhances user confidence and autonomy while reducing support needs, offering a superior balance of efficiency and effectiveness compared to training with fewer cases.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the quiet, dimly lit rooms where radiologists examine brain scans, a new kind of partner has begun to appear. For decades, these specialists have relied on their own trained eyes to spot tiny, subtle signs of disease within the complex architecture of the human brain. Now, artificial intelligence offers to assist, acting as a second set of eyes that can rapidly highlight areas of concern, such as the lesions caused by multiple sclerosis. Yet, introducing a powerful new tool into a high-stakes medical workflow is not as simple as flipping a switch. The challenge lies not just in the technology itself, but in how humans learn to trust and work alongside it. If a doctor is not properly introduced to the software, they may ignore its helpful suggestions or, conversely, rely on it too blindly, potentially missing errors. The key question for hospitals and training programs is how to teach these clinicians to use the tool effectively. Does a quick overview suffice, or does the doctor need to see a variety of examples to truly understand how the system behaves in different situations?
A recent pilot study conducted at the University Hospital of Zurich set out to answer this very question. The researchers wanted to know if the number of practice cases shown to a doctor during training mattered more than the total time spent training. They focused on a specific software tool designed to detect lesions associated with multiple sclerosis on a type of brain scan called a 3D FLAIR MRI. To find the answer, they gathered thirteen radiology clinicians, ranging from those just starting their careers to experienced senior doctors. These volunteers were randomly divided into three groups. Each group received the same ten-minute tutorial on how the software worked, but they differed in how many patient cases they reviewed while learning. One group practiced with a single case, another with two, and the third with three. These cases were carefully chosen to represent the full spectrum of the disease, from mild to severe, ensuring the doctors saw a realistic variety of what the software might encounter.
After the training session, the clinicians moved to an independent testing phase. Without any further instruction, they were asked to review three new, standardized patient cases using the software. During this time, the researchers carefully recorded how long the training took, how many questions the doctors asked while learning, and how many questions they needed to ask while working on their own. They also asked the doctors to rate their own confidence in using the tool on a scale from one to five. The results revealed a clear pattern. The group that trained with only one case finished their onboarding the fastest, in just over twelve minutes. However, when they tried to work independently, they struggled the most, asking an average of more than five questions per case and reporting the lowest level of confidence. The group that trained with two cases took a bit longer to learn but still showed inconsistent results, with some doctors needing significant help and others needing less.
The group that trained with three cases told a different story. Although their initial training session took the longest, clocking in at twenty-six minutes and thirty-eight seconds, they performed the best when working alone. They asked the fewest questions during their independent review, averaging only three and a half questions per case, and they reported the highest confidence levels. The researchers found that the extra fourteen minutes of training time invested by this group paid off by making the doctors more self-sufficient and less reliant on support later on. Interestingly, the study showed that a doctor's years of prior experience did not predict how well they adapted to the new tool. Even the most experienced clinicians needed the same structured exposure to multiple cases as their junior colleagues to feel confident and work efficiently.
The findings suggest that for artificial intelligence tools to be truly useful in medicine, the way they are introduced to users matters deeply. It is not enough to simply show a doctor how the software works once; they need to see a small, diverse set of examples to build a solid mental model of how the system behaves. The study indicates that a slightly longer training session focused on case diversity creates a more capable and confident user than a rushed session focused on speed. While this was a small-scale simulation and not a final proof of how the tool works in every hospital, the results point toward a practical solution: investing a little more time upfront to show doctors a few different scenarios can lead to smoother, safer, and more effective use of artificial intelligence in the future.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.