Similar discrimination, different behaviour under synthetic occlusion: confidence and attribution reproducibility of deep learning models for intraoral ultrasonography
Although multiple deep learning architectures achieve similarly high baseline discrimination for intraoral ultrasonography, they exhibit drastically different and often unreliable behaviors under synthetic occlusion, with EfficientNet-B0 showing severe performance collapse, overconfidence, and poor reproducibility compared to other models.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
Inside the human mouth, a specialized ultrasound probe can capture real-time images of soft tissues without using any radiation. These images reveal layers of muscle and skin, offering a window into anatomy that is useful for planning surgeries or assessing tissue health. However, turning these pictures into reliable diagnoses requires more than just a clear image; it requires a computer program, or model, that can correctly interpret what it sees. In recent years, scientists have trained many such computer models to recognize patterns in medical images. A common way to judge these models is to see how often they get the right answer on a set of test images. But a high score on a standard test does not tell the whole story. It does not reveal how the model behaves when the image is changed, whether it remains confident when it is wrong, or if the reasons it gives for its decision are consistent.
A team of researchers at Yonsei University in South Korea set out to look beyond simple test scores. They focused on a specific task: distinguishing between two regions of the inner cheek, known as the buccal mucosa. One region sits above the cheekbone muscles, and the other sits over the main cheek muscle. While a doctor knows exactly where the probe is placed, a computer model must learn to tell these two areas apart just by looking at the picture. The researchers trained five different types of computer models on hundreds of images taken from healthy volunteers. All five models performed very well on the standard test, correctly identifying the regions in nearly every case. This high performance suggested they were all equally good candidates for future medical use.
To test if they were truly alike, the researchers subjected the models to a series of controlled changes. They took the images and covered up parts of them with a solid, uniform grey block, effectively erasing information from either the top or the bottom half of the picture. This process, known as occlusion, simulates what might happen if part of an image were blurry or missing. When the researchers removed half of the image, four of the five models continued to perform well, showing they could still find the necessary clues in the remaining parts. One model, however, reacted very differently. When half of its image was covered, its ability to distinguish between the two regions collapsed, dropping to a level no better than random guessing.
The most surprising finding was not just that this one model failed, but how it failed. When the other models made mistakes, they tended to be unsure of their answers. The failing model, by contrast, became dangerously confident. Even as its accuracy plummeted to the level of chance, it continued to assign extremely high certainty to its wrong answers. It was as if the model had lost its ability to see the difference but had not lost its belief that it was right. This behavior suggests that a model can appear highly reliable on a standard test while hiding a fragile dependence on specific parts of the image that, when removed, cause it to break down completely.
The researchers also investigated how the models "explained" their decisions. Using a technique that highlights the parts of an image a model focuses on, they checked if these explanations were consistent. They found that the highlighted areas changed significantly depending on how the model was trained, even when the final test scores were the same. This means that a single explanation from one version of a model might look plausible, but it cannot be trusted as a stable guide to what the model is actually learning. The patterns of attention were not reproducible, making it difficult to rely on these visual explanations to understand the model's logic.
The study concludes that two models can look identical on a standard report card yet behave in completely opposite ways when the input is altered. The researchers found that similar baseline performance can conceal a model that is highly sensitive to missing information and prone to confident errors. They did not find a single architectural flaw that caused this, nor did they prove that this specific failure would happen in a real clinical setting with real patients. Instead, they demonstrated that evaluating a model requires more than just checking its accuracy. To truly understand a computer's ability to interpret medical images, one must also test how it handles missing data, how it expresses confidence when it is wrong, and whether its reasoning remains consistent. Without these checks, a model that looks perfect on paper might be unreliable in practice.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.