Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty

1. Problem Statement

Current Large Language Models (LLMs) lack reliable self-awareness regarding the source of their uncertainty. While existing research focuses on "abstention" (refusing to answer), models typically treat all refusals as a generic "I don't know." This coarse approach fails to distinguish between two fundamentally different causes of uncertainty:

• Data Uncertainty: The question itself is ambiguous, underspecified, or lacks critical information (e.g., "James runs 3 sprints... how far?" without specifying sprint distance). The correct action is to request clarification.
• Model Uncertainty: The question has a unique, objective answer, but it exceeds the model's current reasoning or knowledge capabilities (e.g., a complex math problem requiring tools). The correct action is to invoke external tools or admit a capability limit.

The inability to distinguish these sources leads to suboptimal downstream decisions (e.g., asking for clarification on a solvable problem or hallucinating on an ambiguous one) and limits the deployment of reliable, decision-oriented AI agents.

2. Methodology

A. UA-Bench: A New Benchmark

The authors introduce UA-Bench, a benchmark comprising over 3,500 questions designed to evaluate explicit uncertainty attribution.

• Composition: It integrates six datasets spanning Knowledge-Intensive tasks (e.g., GAIA, MuSiQue, SelfAware) and Reasoning-Intensive tasks (e.g., OlympiadBench-math, GSM8K-MiP, MATH-MiP).
• Task Definition: Models are instructed to reason step-by-step and output one of three mutually exclusive tokens:
  1. Answer: If confident and capable.
  2. Data Uncertain: If the input is ambiguous or missing info.
  3. Model Uncertain: If the input is clear but the model cannot solve it.
• Evaluation Metrics:
  • ACC: Standard answer accuracy on answerable questions.
  • DU-F1 (Data-Uncertain F1): Measures the ability to detect ambiguous inputs.
  • MU-F1 (Model-Uncertain F1): Measures the ability to recognize capability limits.
  • AVG-F1: The arithmetic mean of DU-F1 and MU-F1.

B. Empirical Evaluation

The authors evaluated 18 frontier LLMs (including Qwen3, LLaMA-4, GPT-4o/5, Claude Sonnet, Gemini) in both "Non-Thinking" and "Thinking" modes.

• Key Finding: Even State-of-the-Art (SOTA) models struggle to discriminate between the two uncertainty types.
• Paradox: High answer accuracy does not correlate with strong uncertainty attribution. For instance, Qwen3-4B achieved 72.3% accuracy on reasoning tasks but only 23.3% MU-F1.
• Thinking Mode Impact: While "Thinking" modes often improve accuracy, they frequently degrade uncertainty attribution (specifically MU-F1), suggesting that models optimized for reasoning develop a bias that a solution must exist, leading them to misattribute their own failures to data ambiguity.

C. Proposed Solution: RL for Uncertainty Attribution (RL-UA)

To address these gaps, the authors propose a lightweight Reinforcement Learning (RL) strategy using Group Relative Policy Optimization (GRPO).

• Data Synthesis: Instead of using massive human-labeled datasets, they synthesize training data from the dapo-math dataset by creating two variants:
  1. Insufficient Variants: Removing critical info to simulate Data Uncertainty.
  2. Extremely Difficult Variants: Rewriting problems to be solvable in principle but computationally hard for the model to simulate Model Uncertainty.
• Reward Design: A sparse reward function encourages honest self-assessment:
  • +1: Correct answer OR correct uncertainty classification.
  • 0: Incorrect answer but correctly abstained with "Model Uncertain" (honest admission of limits).
  • -1: Hallucination (wrong answer) or misclassification.
• Training: Applied to Qwen3-4B-Instruct and Qwen3-8B (Thinking mode).

3. Key Contributions

1. Conceptual Distinction: Formalized the separation of Data Uncertainty vs. Model Uncertainty, arguing that distinguishing them is critical for adaptive agent behaviors (clarification vs. tool use).
2. UA-Bench: Introduced a comprehensive benchmark for evaluating uncertainty attribution across knowledge and reasoning tasks, moving beyond binary refusal.
3. Empirical Insights: Revealed that current SOTA models cannot reliably distinguish uncertainty sources and that "Thinking" modes can paradoxically worsen self-awareness by inducing overconfidence.
4. RL-UA Method: Demonstrated that a lightweight RL approach trained on synthetic math data can significantly improve uncertainty attribution across different model scales and reasoning styles without sacrificing answer accuracy.

4. Results

• Baseline Performance: On UA-Bench, most models showed high DU-F1 (detecting bad questions) but very low MU-F1 (admitting their own limits). For example, GPT-4o achieved 72.4% AVG-F1 on knowledge tasks but dropped significantly on reasoning tasks where MU-F1 was the bottleneck.
• RL-UA Performance:
  • Qwen3-4B: RL-UA improved MU-F1 on reasoning tasks from 23.3% to 53.5% while maintaining answer accuracy (~73%).
  • Qwen3-8B (Thinking): RL-UA improved MU-F1 from 18.7% to 60.8% and AVG-F1 from 33.0% to 63.5%, with no degradation in accuracy.
• Generalization: Despite being trained only on mathematical tasks, the RL-UA models generalized effectively to knowledge-intensive tasks, proving the method learns a general principle of uncertainty attribution.
• Qualitative Analysis: The RL-trained models corrected systematic failure modes, such as blaming data ambiguity for their own reasoning failures (Case 1) or treating subjective questions as factual gaps (Case 2).

5. Significance

This work shifts the paradigm of LLM safety and reliability from simple "refusal" to principled uncertainty attribution.

• Decision-Oriented AI: By enabling models to distinguish why they don't know, systems can autonomously choose the correct next step: asking a user for missing data (Data Uncertainty) or calling a search engine/calculator (Model Uncertainty).
• Training Efficiency: The paper demonstrates that complex self-awareness can be instilled via lightweight RL on synthetic data, avoiding the need for expensive, large-scale human annotation.
• Future Research: It highlights that current "Thinking" paradigms may inadvertently suppress self-awareness, suggesting a need for new alignment strategies that balance reasoning power with epistemic humility.

The authors conclude that achieving human-level reliability in uncertainty attribution remains an open challenge, but UA-Bench and the RL-UA framework provide a crucial foundation for building transparent and trustworthy AI agents.