Uncertainty-Aware Transformer-Centred Medical Text Classification:A Systematic Methodological Review and Taxonomy
This systematic review of 32 studies on transformer-based medical text classification introduces a four-tier functional taxonomy to organize uncertainty-aware methods, revealing that current research disproportionately focuses on feature representation and training-data reliability while lacking standardized evaluation frameworks for uncertainty estimation and decision-making.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the high-stakes world of healthcare, computers are increasingly asked to read medical notes, scan research papers, and sort through patient records to find answers. These tasks, once the exclusive domain of doctors and researchers, are now being handled by sophisticated software that learns from vast libraries of text. The most powerful tools for this job are based on a technology called the transformer, a type of artificial intelligence that excels at understanding the nuances of human language. When these tools are trained specifically on medical language, they can become incredibly accurate at identifying diseases, categorizing symptoms, or summarizing clinical trials. However, a critical flaw remains: these systems are often too confident. They can look at a confusing or ambiguous medical sentence and declare an answer with absolute certainty, even when the information is weak or the meaning is unclear. In a hospital, such overconfidence can lead to dangerous mistakes, as a computer might present a guess as a fact, potentially influencing a doctor's decision or a patient's treatment plan.
A new systematic review, published in October 2026, investigates how researchers are trying to fix this problem of overconfidence. The study, led by Prabhashrini Dhanushika Manage and colleagues from the Queensland University of Technology and the Australian e-Health Research Centre, does not propose a single new software tool. Instead, it acts as a comprehensive map of the entire field. The researchers gathered and analyzed 32 distinct studies published between 2019 and 2026 that focused on making these medical text classifiers more reliable. Their goal was to understand not just whether the computers were getting the right answer, but how the researchers were teaching the machines to know when they were unsure, and what they did when that uncertainty arose. The team found that while scientists have developed many clever ways to improve the quality of the data the computers read, they have struggled to create a unified way to measure or act upon the computers' doubt.
To make sense of the diverse approaches found in the literature, the authors organized the 32 studies into a four-part framework, or taxonomy, based on where each method intervenes in the classification process. The largest group of studies, accounting for nearly half of all the methods reviewed, focused on the very beginning of the process: feature representation. These researchers worked on improving the raw material fed into the computer. They found ways to enrich the text with medical knowledge, such as linking words to specific medical concepts or using specialized training data that understands the unique vocabulary of hospitals. By strengthening the evidence available to the classifier before it even makes a guess, these methods aim to reduce confusion at the source. While effective at boosting accuracy, the review notes that simply making the input better does not automatically teach the system to recognize its own limits.
A much smaller portion of the research, representing only about one in nine of the methods, focused on the second tier: explicitly measuring uncertainty. These studies developed techniques to ask the computer, "How sure are you?" and to assign a numerical score to that feeling of doubt. Some methods looked at how much the computer's internal logic wavered, while others compared the computer's confidence against its actual track record of correctness. The review highlights that this area is underdeveloped compared to the others. Because these studies used different ways to measure doubt and different standards for what counts as "reliable," it is difficult to compare their results directly. There is no single, agreed-upon way to say that one computer is more cautious than another, which makes it hard to build a universal safety net.
The third tier of the framework deals with what happens after the computer has made a prediction and assessed its own confidence. This is where uncertainty-aware decision making comes in. In these studies, the computer does not just output a label; it uses its level of certainty to decide its next move. If the system is highly confident, it might proceed automatically. If it is unsure, it might route the case to a human doctor for review, reject the input entirely, or pause to gather more information. The researchers found that while these strategies are promising, they are currently very specific to the task at hand. A rule that works for deciding when to trust a diagnosis of pneumonia might not work for sorting cancer research papers. The review points out that there is no established, cross-setting rule for when a computer should step back and let a human take over, leaving a gap in how these systems are deployed in real-world scenarios.
The final and second-largest category of research focused on the quality of the training data itself. This tier, known as training-data reliability, addresses the problem of how the computers learn in the first place. Many of the studies in this group worked on filtering out bad examples, creating synthetic data to fill gaps, or refining weak labels that were generated by other computers. For instance, some researchers used advanced language models to generate new medical sentences to help train the system, while others developed methods to check if those generated sentences were medically accurate before letting the system learn from them. The review found that while these methods are essential for handling the messy reality of medical records, there is no shared standard for determining when a piece of training data is good enough to be trusted. Without a common way to validate the evidence the computer learns from, it remains difficult to guarantee that the system's knowledge base is solid.
The authors of the review conclude that the path to truly reliable medical text classification requires a coordinated effort across all four of these areas. Currently, the field is imbalanced, with a heavy focus on improving the input data and the training process, but a relative lack of attention on how to measure and act upon the computer's uncertainty. The study suggests that simply making a computer more accurate is not enough; a system must also be able to recognize when it is guessing and have a clear protocol for what to do in those moments. The researchers call for future work to establish consistent ways to evaluate uncertainty, to test decision rules across different medical settings, and to create transparent standards for verifying training data. Until these pieces come together, the most reliable approach remains one where human oversight is maintained, ensuring that the computer's confidence is always checked against human judgment.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.