🤖 machine learning

LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy

This article proposes Adaptive Conformal Semantic Entropy (ACSE), a novel method that quantifies the uncertainty of LLMs by adaptively measuring semantic dispersion across diverse responses and applying conformal calibration to provide finite-sample error guarantees without distributional assumptions, thereby surpassing existing lexical and probabilistic baseline methods in safety-critical applications.

Original authors: Hamed Karimi, Vaishali Meyappan, Reza Samavi

Published 2026-05-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hamed Karimi, Vaishali Meyappan, Reza Samavi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you ask a very confident, well-informed librarian (the AI) a question. The librarian might answer with absolute certainty, even if they are completely wrong. This is the "overconfidence" problem that the work addresses: Large Language Models (LLMs) often hallucinate (invent things) while sounding 100% sure.

The work introduces a new safety system called ACSE (Adaptive Conformal Semantic Entropy). Think of ACSE as a "reality check" mechanism that listens not only to what the librarian says but also checks whether the librarian is actually certain about the meaning of their answer.

Here is how it works, broken down into simple steps:

1. The Problem: The "Synonym Trap"

Current safety checks often look at the specific words the AI chooses. If the AI says "The capital is Sydney" 70% of the time and "The capital is Canberra" 30% of the time, a simple word counter might think: "Oh, it is quite sure it's Sydney!" and give the green light.

But here is the catch: The AI might be confident about the wrong answer (Sydney) while being uncertain about the correct answer (Canberra). Or it might give five different answers that all mean the same thing (e.g., "Sydney", "Syd", "The big harbor city"). A simple word counter gets confused by these variations and thinks the AI is uncertain, even though it is merely being chatty.

2. The Solution: The "Group Hug" Method (Semantic Clustering)

ACSE changes the game by asking the AI to answer the same question ten times.

Step A: It takes these ten answers and translates them into "meaning maps" (embeddings).
Step B: It groups these answers into "neighborhoods" based on their meaning, not their spelling.
- Example: If 9 answers say "Sydney" and 1 says "Canberra", they form two distinct neighborhoods.
- Example: If 5 answers say "Sydney" and 5 say "The capital is Sydney", they are all "hugged" into the same neighborhood because they mean the same thing.

3. The "Brittleness" Detector (Adaptive Inflation)

This is the secret of the work. Just because the AI agrees on one answer (like "Sydney") does not mean that answer is safe.

The Analogy: Imagine a group of people agreeing on a direction. If they are all standing in a tight, solid circle, that is a strong consensus. But if they agree on a direction while standing on shaky, trembling ground, that is a brittle consensus.
ACSE looks for this "trembling." It checks whether the group agreeing on "Sydney" is actually shaky (perhaps the answers are slightly different or the group is very small).
If the group is "brittle," ACSE inflates the uncertainty value. It essentially says: "Even though you all agree, your agreement is shaky, so I will treat this as a high-risk situation."

4. The "Safety Net" (Conformal Calibration)

Finally, the system must know exactly when to say "I don't know" (abstain) and when to give an answer.

The authors use a statistical "safety net" called Conformal Prediction.
They first test the system on a set of practice questions. They determine a "stopping threshold."
The Rule: If the "trembling value" (uncertainty) is below the line, the AI answers. If it is above, the AI stays silent.
The Guarantee: This is not a guess. The mathematics guarantees that if you set the safety net to catch 90% of errors, it will catch at least 90% of errors, no matter what the AI does. It promises that the errors you actually see will be rare.

The Results: Why It Matters

The work tested this on various AI models and datasets (such as quiz questions).

The Competition: Old methods (like counting word probabilities) were like a shaky compass. They often gave high confidence to wrong answers.
The Winner: ACSE acted like an intelligent navigator. In a quiz test, it correctly identified wrong answers 88% of the time (AUROC 0.88), while the next best method only reached 80%.
The Safety: It successfully prevented the AI from giving wrong answers in high-risk situations much more often than previous methods, without being so cautious that it stopped giving answers altogether.

In Brief

ACSE is a system that asks an AI to answer a question multiple times, groups the answers by meaning rather than words, checks whether the group is standing on solid or shaky ground, and uses a mathematically proven safety net to decide when to speak up and when to stay silent. It ensures that when the AI speaks, it is not only confident but actually reliable.

Technical Summary: Adaptive Conformal Semantic Entropy (ACSE)

Problem Statement
Large Language Models (LLMs) frequently exhibit excessive overconfidence, particularly when generating hallucinations, posing significant risks for deployment in safety-critical domains such as healthcare, law, and scientific research. Existing Uncertainty Quantification (UQ) methods rely primarily on token-level signals, such as next-token entropy or sequence log-likelihood. These approaches fail to capture semantic variance; they often assign high confidence to outputs that are lexically diverse but semantically inconsistent or incorrect. Furthermore, newer semantic-level methods, such as Semantic Entropy (SE), rely on hard clustering, which can lead to unstable estimates by ignoring overlapping semantic regions. Other conformal approaches, while offering statistical guarantees, often treat uncertainty as a scalar signal, making them susceptible to "false-consensus" pitfalls where lexically consistent but factually incorrect clusters meet confidence thresholds.

Methodology
The paper proposes Adaptive Conformal Semantic Entropy (ACSE), a model-agnostic framework for estimating prompt-level uncertainty by measuring semantic dispersion in LLM outputs. The methodology proceeds in three main phases:

Semantic Embedding and Soft Clustering:
For a given prompt, the model generates $n$ diverse responses using Nucleus Sampling. These responses are embedded into a continuous semantic vector space using a sentence encoder. Instead of hard clustering, ACSE employs Hierarchical Agglomerative Clustering (HAC) with cosine-based distance to form semantic groups, followed by soft cluster assignments. This allows responses to probabilistically belong to multiple clusters, preserving signals of semantic ambiguity. A base uncertainty value, $u(x)$ , is derived from the normalized entropy of the resulting cluster distribution.
Adaptive Uncertainty Inflation:
To address structural fragility in clusters (e.g., weak support for dominant clusters or high internal diversity), ACSE introduces an adjusted uncertainty value, $\hat{u}(x)$ . This value inflates the base Semantic Entropy based on five prompt-level robustness features:
- Semantic Entropy: Measures multimodality.
- Centroid Distance: Evaluates the geometric support of the dominant response.
- Dominant Cluster Dispersion: Quantifies internal coherence.
- Dominant Cluster Size: Penalizes fragile consensus supported by few samples.
- Distance to Threshold: Suppresses unjustified confidence in low-uncertainty regimes.
  These features are aggregated into a "fragility measure" $B(x)$ , which scales an inflation factor $\lambda(x)$ . The final value $\hat{u}(x)$ is a bounded, monotonic transformation that increases uncertainty when cluster structures indicate risk.
Conformal Calibration:
The adjusted values are calibrated on a held-out set of prompts using Conformal Prediction (CP). This establishes a decision rule with finite-sample, distribution-free guarantees. The system serves two functions:
- Prompt-level Decision: Determines whether a response is accepted or rejected based on a threshold, ensuring the error rate among accepted responses remains below a user-specified tolerance $\alpha$ .
- Response-level Prediction Sets: Constructs a set of sampled responses that satisfy conformal coverage, supporting user decision-making by identifying semantically representative and conformal outputs.

Main Contributions

ACSE Framework: A novel method for estimating entropy-based uncertainty at the semantic level, going beyond token-level signals to capture dispersion in meaning.
Adaptive Inflation Mechanism: An adjusted uncertainty value that leverages cluster robustness features to adaptively inflate semantic uncertainty, explicitly penalizing semantically ambiguous response semantics, and mitigating hallucinations.
Conformal Guarantees: A post-hoc calibration phase that learns thresholds for both prompt-level rejection and response-level prediction sets, providing formal guarantees for error rates.
Empirical Validation: Comprehensive experiments demonstrating superior discrimination and calibration compared to state-of-the-art baselines.

Experimental Results
The authors evaluated ACSE across five benchmarks (TriviaQA, CoQA, Natural Questions, TruthfulQA, and MMLU) using various LLM architectures (Mistral-7B, LLaMA-2, Falcon, Qwen).

Hallucination Detection: ACSE consistently outperformed baselines in discrimination metrics. On the TriviaQA dataset, ACSE achieved an AUROC of 0.88, significantly surpassing the Token Entropy baseline (0.65) and the Conformal Abstention Policy (CAP) (0.80).
Safety Metrics: ACSE demonstrated lower False Positive Rates (FPR) at high recall thresholds. For instance, with Falcon-7B, ACSE reduced the FPR@95 from 0.48 (CAP) to 0.31, corresponding to a relative decrease in accepted hallucinations of 35.4%.
Conformal Guarantees: ACSE strictly adhered to user-specified coverage levels ( $\alpha$ ) while maintaining higher acceptance rates than competing methods. At $\alpha=0.10$ , ACSE achieved an acceptance rate of 75.8% compared to 65.4% for CAP, with smaller average prediction set sizes (1.07 vs. 1.32) and superior calibration stability (SSCV).
Uncertainty Quantification: Visual analysis confirmed that ACSE effectively separates correct answers from hallucinations, assigning high uncertainty to hallucinations even when baseline methods (such as SE or Token Entropy) showed high confidence.

Significance
The paper argues that ACSE offers a robust solution for the reliable deployment of LLMs in safety-critical domains by addressing the specific failure modes of existing UQ methods: the inability to distinguish superficial lexical variation from genuine semantic ambiguity, and susceptibility to false-consensus pitfalls. By combining semantic dispersion analysis with adaptive inflation and conformal guarantees, ACSE provides a statistically rigorous mechanism for detecting hallucinations and controlling error rates without requiring model retraining. The authors posit that the computational costs arising from sampling and post-processing are justified in high-risk applications where model reliability is paramount.