Post-hoc reliability assessment and visual explanation for deep learning-based oral mucosal lesion detection
This paper presents a post-hoc framework that enhances the reliability and interpretability of a frozen YOLOv11 detector for oral mucosal lesions by analyzing feature embedding densities to identify untrustworthy predictions and using nearest-neighbor retrieval to provide visual explanations, all without requiring model retraining.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of medical imaging, artificial intelligence has become a powerful tool for spotting diseases that the human eye might miss. These systems are trained on thousands of photographs, learning to recognize patterns that signal illness. However, a significant problem remains: when these computer programs make a prediction, they often provide a confidence score that can be misleading. A system might declare a diagnosis with 99 percent certainty while looking at an image that is completely different from anything it has ever seen before. In safety-critical fields like medicine, where a wrong guess can delay life-saving treatment, knowing when to trust the machine is just as important as the machine's ability to see. The challenge is to find a way to check the reliability of a prediction without having to rebuild the entire system from scratch.
Researchers at several German and international institutions have tackled this issue by developing a new method to audit the decisions of an artificial intelligence designed to detect oral mucosal lesions. These are conditions affecting the soft tissues inside the mouth, ranging from harmless white patches to potentially cancerous growths. The team focused on a specific type of AI detector called YOLO, which is excellent at finding objects in images and drawing boxes around them. While the detector was already trained to identify three types of lesions and healthy tissue, the researchers wanted to understand the "why" behind each decision and determine if the AI was truly confident or just guessing. They did not retrain the model or change its internal code. Instead, they looked at the mathematical fingerprints, known as embeddings, that the AI created for every image it analyzed. These fingerprints represent the visual features the AI used to make its call.
The researchers took these high-dimensional fingerprints and mapped them onto a simple two-dimensional landscape. On this map, images that look very similar to each other cluster together, while different types of lesions form their own distinct groups. By analyzing how crowded or sparse a specific area was on this map, they could calculate a reliability score. If an AI's prediction landed in a dense cluster of similar, correctly identified training images, the system considered the prediction reliable. If the prediction landed in a sparse, empty area far from any known examples, the system flagged it as uncertain. This approach allowed them to assess the trustworthiness of every single detection without altering the original detector. They found that this density-based score was far better at identifying uncertain predictions than the standard confidence numbers the AI usually provides. In tests involving over 3,500 images, this method successfully distinguished between reliable and unreliable predictions with an AUC ranging from 0.92 to 0.95 across the different disease categories.
To make these abstract scores useful for doctors, the team added a visual explanation tool. When the system flagged a prediction as uncertain, it could instantly retrieve and display the four most similar images from its training database. This created a mosaic showing the doctor exactly what the AI was comparing the current patient to. For example, if the AI misidentified a specific type of white patch, the visual explanation would show that it was comparing the patient to images of a different, but visually similar, condition. This allowed clinicians to see the AI's reasoning process in action. The study revealed that the AI often struggled with lesions that looked alike, such as certain forms of leukoplakia and lichen planus, which share overlapping visual features. In these borderline cases, the new method correctly identified that the AI was operating in a zone of ambiguity, whereas the standard confidence score would have remained deceptively high.
The researchers also tested the system on images of lesions that were not part of the original training set, such as blistering diseases or frictional keratoses. They found that the AI sometimes assigned these unknown conditions to the closest known category, and the density score did not always flag them as completely unknown. This highlights a crucial limitation: the system measures how well a prediction fits the training data, not whether the image represents a completely new disease. However, the visual retrieval tool proved valuable here as well. Even when the AI made a mistake, the displayed training neighbors helped a human expert realize that the visual similarity was misleading and that the diagnosis required more than just a photograph. The study concludes that by combining a density-based reliability check with visual examples, medical professionals can better understand when to trust an AI's diagnosis and when to seek a second opinion, all without needing to retrain the complex models already in use.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.