💬 NLP

SpanUQ: Span-Level Uncertainty Quantification for Large Language Model Generation

The paper introduces SPANUQ, a lightweight framework that addresses the limitations of token- and sequence-level uncertainty estimation by employing a DETR-style decoder to simultaneously detect semantically coherent text spans and quantify their uncertainty via a Mixture of Beta distribution, achieving superior error localization and efficiency across multiple large language models.

Original authors: Yimeng Zhang, Yingying Zhuang, Ziyi Wang, Yuxuan Lu, Pei Chen, Aman Gupta, Zhe Su, Ming Tan, Zhilin Zhang, Qun Liu, Manikandarajan Ramanathan, Rajashekar Maragoud, Edward Vul, Jing Huang, Dakuo Wang

Published 2026-07-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yimeng Zhang, Yingying Zhuang, Ziyi Wang, Yuxuan Lu, Pei Chen, Aman Gupta, Zhe Su, Ming Tan, Zhilin Zhang, Qun Liu, Manikandarajan Ramanathan, Rajashekar Maragoud, Edward Vul, Jing Huang, Dakuo Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you ask a very smart but occasionally hallucinating AI a question, like "Who is Marie Curie?" The AI gives you a long, confident-sounding answer. But somewhere in that answer, it makes a mistake (e.g., saying she won a Nobel Prize in 1901 instead of 1903).

The problem with current AI safety tools is that they are bad at finding exactly where that mistake is. This paper introduces a new tool called SPANUQ that acts like a high-tech "fact-checking highlighter" for AI answers.

Here is a simple breakdown of how it works and why it's different:

1. The Problem: Too Blurry or Too Noisy

Think of checking an AI's answer like grading a student's essay.

The Old Way (Token-Level): Imagine a teacher who puts a red mark under every single word in the sentence. They mark "the," "was," and "a" with a score. This is too noisy. It doesn't tell you which specific idea is wrong.
The Other Old Way (Sequence-Level): Imagine a teacher who looks at the whole essay and gives it one single grade, like a "C." This tells you the essay has problems, but it doesn't tell you which paragraph or which sentence is the issue. You can't fix it if you don't know where the error is.

2. The Solution: The "Highlighter" Approach (Span-Level)

The authors realized that the natural unit of meaning isn't a single word, and it isn't the whole paragraph. It's a "Span."

A Span is a chunk of text that holds one complete idea (like "Polish physicist" or "won the Nobel Prize in 1901").
SPANUQ is designed to find these specific chunks and give each one its own "confidence score."
The Result: Instead of a single grade for the whole essay, SPANUQ highlights "Polish physicist" (Green/Confident), "won the Nobel Prize" (Green/Confident), and "1901" (Red/Very Uncertain). This tells you exactly where to look.

3. How It Works: The "Lightweight Detective"

Usually, to find errors, you have to ask the AI the same question 20 times and compare all the answers to see where they disagree. This is slow and expensive (like hiring 20 detectives to solve one case).

SPANUQ is different:

The Detective: It's a tiny, lightweight add-on (only about 25 million parameters, which is tiny compared to the massive AI model it helps).
The Trick: It looks at the AI's "brain waves" (hidden states) while the AI is thinking. It has been trained to recognize the subtle patterns that indicate uncertainty, effectively "distilling" the knowledge of those 20 slow checks into a single, instant glance.
The Speed: It is 10 to 20 times faster than the old methods because it only needs to run the AI model once.

4. The "Training School" (SPANUQ-BENCH)

To teach this detective, the authors built a massive training school called SPANUQ-BENCH.

They generated 20,000 questions and answers.
They used a "gold standard" method (asking the AI 20 times and checking against Wikipedia) to create "answer keys" that tell them exactly which parts of the text were wrong and how wrong they were.
They trained the detective on 293,000 of these specific text chunks.

5. The Results: Why It Wins

When they tested SPANUQ against other methods:

Accuracy: It was much better at spotting the exact wrong chunks (achieving a score of ~0.94 out of 1.0).
Localization: It found the errors 39% better than the best "heuristic" (rule-of-thumb) methods.
Versatility: It worked well on different sizes of AI models, from small ones to very large ones.

The Bottom Line

The paper claims that SPANUQ is the first tool that can instantly look at an AI's answer, break it down into meaningful ideas, and tell you exactly which idea is shaky and how shaky it is. It does this without needing to run the AI multiple times, making it fast enough to use in real-time applications where you need to trust the AI's output.

Important Note: The authors warn that while this tool is great at spotting uncertainty, it doesn't guarantee the AI is telling the truth. It just tells you, "Hey, this specific part looks risky, so you should double-check it." It is a tool to help humans, not a replacement for human judgment.

Technical Summary: SPANUQ – Span-Level Uncertainty Quantification for Large Language Model Generation

1. Problem Formulation

Current approaches to uncertainty estimation in Large Language Models (LLMs) operate at suboptimal granularities, failing to balance semantic coherence with error localization.

Token-level methods (e.g., predictive entropy, token probes) assign scores to individual tokens. These are often semantically incomplete; function words receive meaningless scores, and single tokens rarely constitute a verifiable fact.
Sequence-level methods (e.g., semantic entropy, multi-sample consistency) produce a single score for an entire response. While they capture meaningful semantic uncertainty, they cannot localize which specific parts of a generation are unreliable, often averaging correct and incorrect claims into an unactionable middle ground.

The authors formalize Span-Level Uncertainty Estimation (SLUE) as a new task. A "span" is defined as a contiguous text segment conveying a single, coherent unit of meaning (e.g., an atomic fact or claim) whose uncertainty can be independently assessed. The goal is to jointly detect these variable-length spans and assign continuous uncertainty scores to each from a single forward pass.

2. Methodology: SPANUQ

The paper introduces SPANUQ, a lightweight probe (~25M parameters) that distills uncertainty knowledge from expensive multi-sample inference into a single forward pass over frozen LLM hidden states.

Architecture

SPANUQ consists of five key components:

Multi-Layer Fusion: Hidden states from selected layers (typically mid-to-late layers where factual knowledge peaks) are fused via mean pooling.
Token Encoder: A 2-layer Transformer encoder contextualizes the fused states into a content pool.
DETR-Style Span Decoder: Inspired by DETR, the model uses a set of learnable span queries to attend to the token sequence. This allows for parallel detection of overlapping spans, a limitation of traditional BIO tagging.
Span Feature Enrichment: A differentiable soft boundary mask extracts content features from the predicted span regions, fusing them with the query embeddings via a gated residual connection. This ensures the uncertainty scorer has direct access to the span's semantic content.
Prediction Heads:
- Regression: Predicts normalized start and end boundaries.
- Validity Classification: Distinguishes real spans from unused query slots.
- Mixture of Beta (MoB): Models uncertainty as a mixture of Beta distributions ( $J=3$ components). This captures the bimodal nature of uncertainty (spans are either highly confident or highly uncertain) and allows for U-shaped distributions.

Training and Refinement

Two-Phase Training: The model first undergoes a warmup phase to establish reliable span boundaries, followed by a joint phase optimizing all components.
Loss Functions: The training objective combines span regression (L1 + Generalized IoU), validity classification, and uncertainty estimation. The uncertainty loss includes a Beta NLL (Negative Log-Likelihood) and a contrastive ranking loss to preserve the ordinal structure of uncertainty between spans.
Uncertainty-Conditioned Iterative Refinement (UCIR): During inference, initial estimates are fed back into the decoder for a refinement round. This "second look" allows the model to correct systematic biases based on its own initial predictions, adding minimal overhead (<15%).

3. Benchmark: SPANUQ-BENCH

To evaluate SLUE, the authors constructed SPANUQ-BENCH, the first span-level uncertainty benchmark.

Scale: 20,000 prompts across five domains (Long-form QA, TriviaQA, ELI5, Biography, FELM) and ~293,000 annotated spans.
Labeling Strategy: Ground-truth soft labels ( $u^* \in [0, 1]$ ) are derived via multi-sample distillation. For each prompt, 20 diverse responses are sampled, decomposed into atomic claims, and verified against Wikipedia. The uncertainty label for a span is the fraction of samples where the claim is unsupported.
Evaluation: The benchmark tests discrimination (AUROC), regression accuracy (MAE), and ranking quality (Spearman correlation).

4. Key Results

Experiments were conducted on five LLM backbones (Qwen3 series and Mistral-7B) ranging from 4B to 30B parameters, including both dense and Mixture-of-Experts (MoE) architectures.

Span-Level Performance: SPANUQ consistently achieves the best span-level uncertainty quality across all models.
- AUROC: 0.908 – 0.944 (outperforming the strongest probe baseline by ~5.8 points and sampling-based methods by >13 points).
- MAE: 0.110 – 0.129.
- Efficiency: It is 10–20× faster than sampling-based methods (e.g., SelfCheckGPT, Semantic Entropy) as it requires only a single forward pass.
Span Detection: The DETR-based detector achieves an F1 score of 0.910, surpassing the best heuristic baseline (Sliding Window) by 39.4%. It successfully handles overlapping spans, whereas BIO taggers fail significantly (F1 = 0.079) due to non-overlapping constraints.
Sequence-Level Decomposability: The authors observe that sequence-level uncertainty is partially decomposable into span-level components. The learned importance-weighted span composition achieves a sequence-level Spearman correlation ( $\rho_{seq}$ ) of 0.839, suggesting that span-level estimation subsumes sequence-level estimation as a special case.
Ablation Findings:
- Mixture of Beta (MoB): Crucial for capturing multi-modal uncertainty, reducing ECE by 62% compared to a single Beta distribution.
- UCIR: Provides the second-largest gain in ranking quality ( $\rho_{span}$ ), effectively correcting systematic errors.
- Regression vs. Pointer: Continuous regression for boundary prediction outperforms cross-attention pointer mechanisms by 3.4% in AUROC.

5. Significance and Claims

The paper claims that SPANUQ bridges the gap between token-level and sequence-level uncertainty estimation by operating at the "natural granularity" of verifiable claims.

Interpretability and Actionability: Unlike sequence-level scores, SPANUQ provides interpretable, localizable uncertainty scores for specific claims (e.g., identifying "1901" as uncertain in a sentence about Marie Curie, while confirming "Polish physicist").
Generalization: The framework generalizes across different model families, scales, and architectures (dense vs. MoE) without retraining the backbone.
Practical Deployment: By distilling multi-sample uncertainty into a single forward pass, SPANUQ offers a computationally feasible solution for high-stakes domains (healthcare, legal, research) where identifying specific hallucinations is critical.

The authors note limitations: SPANUQ requires white-box access to hidden states, relies on a compute-intensive one-time label construction pipeline, and currently focuses on factual uncertainty in English. They emphasize that while SPANUQ estimates epistemic uncertainty from hidden-state patterns, it does not guarantee factual correctness and should complement, not replace, human judgment.