Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models

In the rapidly evolving landscape of artificial intelligence, large language models have become powerful tools for generating text, answering questions, and assisting with complex tasks. However, a persistent challenge remains: these systems sometimes produce confident-sounding answers that are factually incorrect, a phenomenon known as hallucination. For users who rely on these models for critical decisions, the danger lies not just in being wrong, but in being wrong with absolute certainty. To address this, researchers have long sought ways to measure a model's uncertainty, essentially asking the system how sure it is of its own output. Traditional methods often provide a single confidence score for an entire response, much like a weather forecast giving one probability for a whole day. Yet, a single answer can contain a mix of accurate facts and subtle errors, making a single overall score too blunt to be useful for someone who needs to verify specific details before acting.

A new study by Toghrul Abbasli and colleagues at Tsinghua University and other institutions proposes a more granular approach to this problem. Instead of treating a response as a single block of text, the researchers developed a method to break down every answer into its smallest, independent units of information, which they call atomic claims. They then assign a specific confidence score to each of these individual claims. The team tested this technique on several modern language models, including systems from major developers, using two distinct types of question sets: one focused on general factual knowledge and another designed to trick models into accepting false premises. Their findings suggest that by isolating and calibrating confidence at the claim level, it is possible to create a much more reliable signal for decision-makers. This approach allows users to accept the parts of an answer that are highly likely to be true while flagging specific sentences for further review or rejection, offering a practical path toward safer and more trustworthy artificial intelligence in high-stakes environments.

The core of the researchers' work involves a multi-step pipeline that operates without needing to modify the internal workings of the language models themselves. First, the system generates an answer to a user's question. Next, an auxiliary tool breaks this answer down into short, self-contained statements, such as "Einstein proposed the theory of relativity" or "This occurred in 1905." Once the answer is decomposed, the system evaluates each statement individually. It uses two primary signals to determine how confident the model should be in each claim. The first signal comes from the model's own self-evaluation or verbalized confidence, where the model is prompted to state how sure it is. The second signal relies on consistency; the system generates multiple variations of the answer and checks if the same claim appears in all of them. If a claim is repeated across different attempts, it is treated as more reliable. These two signals are then combined to produce a final confidence score for each specific claim.

To ensure these scores are meaningful, the researchers applied a statistical process called post-hoc calibration. This step adjusts the raw confidence numbers so that they align with reality. For instance, if the system assigns a confidence of 90 percent to a set of claims, calibration ensures that roughly 90 percent of those claims are actually correct. The study tested this framework on six different language models, ranging from open-source systems to proprietary commercial models, across thousands of questions. The results showed that this claim-level approach significantly reduced the error rate in confidence estimation compared to methods that only looked at the response as a whole. On factual questions, the calibrated scores became much more accurate, meaning that when the system said it was 90 percent sure, it was indeed correct about 90 percent of the time. This precision allows a human user to make a concrete decision: they can trust the high-confidence claims immediately while directing their attention to the low-confidence ones for verification.

However, the study also identified a clear boundary where this method faces limitations. When the questions were designed with false premises—traps that assume something untrue is a fact—the models often generated fluent, confident, but entirely incorrect claims. In these adversarial scenarios, the claim-level confidence scores sometimes failed to detect the error because the model was internally consistent in its mistake. The researchers found that while their method excelled at sorting out mixed answers containing both right and wrong information, it could not always overcome a model that was confidently wrong about a foundational fact. In such cases, the study suggests that confidence scores alone are insufficient; an external check, such as a search for evidence or a cross-examination by another system, is necessary to catch these deep-seated errors.

The practical implication of this work is a shift in how we might interact with artificial intelligence in professional settings. Rather than accepting or rejecting an entire response based on a vague sense of trust, decision-makers can now rely on a detailed map of reliability. A medical report generated by an AI, for example, could be accepted for its general structure while specific drug dosages or patient history details are flagged for human review based on their individual confidence scores. The researchers emphasize that this technique works even with "closed-box" models, where the internal mechanics are hidden, making it applicable to the most advanced commercial systems currently in use. While the method does not solve the problem of hallucination entirely, particularly when models are tricked by false assumptions, it provides a robust tool for managing uncertainty. By breaking down the black box of a language model's output into verifiable pieces, the study offers a way to make high-stakes decisions with a clearer understanding of what can be trusted and what requires further investigation.