💬 NLP

Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Tasks

This paper introduces the Refusal Index (RI), a novel metric based on the correlation between refusal and error probabilities, to effectively measure and reveal the often unreliable knowledge-aware refusal capabilities of Large Language Models in factual tasks.

Original authors: Wenbo Pan, Jie Xu, Qiguang Chen, Junhao Dong, Libo Qin, Xinfeng Li, Haining Yu, Xiaohua Jia

Published 2026-05-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wenbo Pan, Jie Xu, Qiguang Chen, Junhao Dong, Libo Qin, Xinfeng Li, Haining Yu, Xiaohua Jia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The Overconfident Student

Imagine a student taking a very difficult history test. A truly smart student knows when they don't know an answer and will raise their hand to say, "I'm not sure about this one." However, Large Language Models (LLMs) are often like a student who is overconfident. Even when they have no idea what the answer is, they will confidently make up a story (a "hallucination") rather than admit they don't know.

This is dangerous because if you ask an AI for medical or legal advice, you need it to say "I don't know" if it's unsure, rather than confidently giving you wrong information.

The Old Way of Measuring: Counting Mistakes

Previously, researchers tried to measure how good an AI is at this by looking at two simple numbers:

How often it refuses to answer.
How often it gets the answer right.

The paper argues these methods are flawed. It's like judging a student only by how many times they raised their hand to say "I don't know."

If you tell the student, "Please say 'I don't know' to everything," they will raise their hand 100% of the time. But that doesn't mean they are smart; it just means they are following orders.
If you tell them, "Answer everything no matter what," they might get more questions right, but they will also give wrong answers confidently.

The old metrics couldn't tell the difference between a student who wisely refuses hard questions and one who is just randomly refusing or answering.

The New Solution: The "Refusal Index" (RI)

The authors created a new score called the Refusal Index (RI). Think of this as a "Truth-Telling Score."

Instead of just counting how many times the AI says "I don't know," the RI asks: "Does the AI say 'I don't know' specifically when the question is hard, and does it answer when the question is easy?"

High RI: The AI acts like a wise librarian. It refuses the obscure, impossible questions but happily answers the easy ones. It knows the difference.
Low RI: The AI acts like a confused robot. It might refuse easy questions it could answer, or it might confidently answer impossible questions. Its "refusal" behavior is random or broken.

How They Measured It: The "Two-Pass" Trick

To calculate this score without needing to know the AI's internal "thoughts" (which we can't see), the researchers used a clever Two-Pass Trick:

Pass 1 (The Honest Test): They ask the AI a bunch of questions and let it decide: "Do you want to answer, or do you want to say 'I don't know'?" They record which ones it refused.
Pass 2 (The Forced Answer): They take only the questions the AI refused in the first round. They go back and say, "Okay, you must answer these now, no skipping allowed." They see if the AI actually could have gotten them right if it had tried.

By comparing the two rounds, they can see if the AI was smart enough to refuse the questions it couldn't answer. If the AI refused the hard ones in Pass 1, but got them wrong in Pass 2, it has a High Refusal Index.

What They Discovered

The researchers tested 16 different AI models (like GPT-4, Claude, Llama, etc.) and found some surprising things:

Accuracy Doesn't Guarantee Wisdom: Just because an AI is very good at answering questions (high accuracy) doesn't mean it knows when to stop. Some very smart models still confidently guess on things they don't know.
The "Family" Matters Most: The most important factor in whether an AI knows when to refuse isn't how big it is or how many questions it gets right. It's which company made it. Some AI "families" (like Claude and Qwen) consistently act more wisely than others (like Gemini or GPT-4.1), regardless of their size.
Context is Key: If you give the AI a document to read and ask a question that isn't in the document, the AI gets confused. It often fails to refuse, trying to guess based on its own training data instead of sticking to the document.

The Bottom Line

The paper concludes that we need a new way to judge AI. We shouldn't just ask, "How many questions did you get right?" We also need to ask, "Did you know when to stop and say 'I don't know'?"

The Refusal Index is the new ruler for this. It helps us find AI models that are not just smart, but also humble and reliable enough to admit their limits.

Technical Summary: Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Tasks

Problem Statement

Large Language Models (LLMs) are increasingly deployed for knowledge-intensive factual tasks. However, they frequently suffer from poor calibration, providing incorrect answers with high confidence (hallucinations). A critical safety mechanism is knowledge-aware refusal: the ability of a model to decline answering questions that fall outside its knowledge boundaries. While recent work has explored inducing refusals via prompting or fine-tuning, existing evaluation metrics fail to accurately measure this capability. Traditional metrics, such as refusal rates or combinations of accuracy and refusal (e.g., F-score), are often heuristic, sensitive to prompt-induced changes in refusal frequency, and fail to capture the fundamental correlation between a model's decision to refuse and the actual incorrectness of the answer it would have provided.

Methodology: The Refusal Index (RI)

To address these gaps, the authors propose the Refusal Index (RI), a novel metric designed to quantify knowledge-aware refusal.

Definition

The RI is defined as the Spearman rank correlation between a model's refusal probability ( $r_i$ ) and its error probability ( $w_i$ ) across a set of questions.
$\text{RI} = \rho_S = \text{Corr}(\text{Rank}(r_i), \text{Rank}(w_i))$
A perfect RI of 1 indicates that a model consistently refuses the most difficult (error-prone) questions and answers the easiest ones. An RI of 0 implies random refusal behavior. Unlike calibration metrics that measure absolute discrepancies (e.g., Expected Calibration Error), RI focuses on the rank of difficulty, making it robust to shifts in overall refusal rates.

Two-Pass Evaluation Framework

Since refusal probabilities are latent (a model either refuses or answers in a single pass), the authors introduce a lightweight two-pass evaluation method to estimate RI:

First Pass: The model is evaluated on a factual dataset with a system prompt allowing it to answer or refuse. Responses are classified as Correct, Incorrect, or Refused. This yields the refusal rate ( $r$ ) and the accuracy of attempted answers.
Second Pass: For questions refused in the first pass, the system prompt is updated to force the model to answer (removing the refusal option). The correctness of these forced answers is recorded.
Computation: Using the joint distribution of refusal decisions and forced-answer correctness, the authors model the relationship using a Gaussian copula. They estimate the correlation parameter ( $\rho$ ) via maximum likelihood on the resulting $2 \times 2$ contingency table and convert it to the Spearman rank correlation ( $\rho_S$ ) using the standard formula: $\rho_S = \frac{6}{\pi} \arcsin(\frac{\rho}{2})$ .

Key Contributions

Refusal Index (RI): A principled metric that directly measures the alignment between refusal decisions and answer incorrectness, independent of a model's overall accuracy or refusal rate.
Lightweight Evaluation: A two-pass protocol that requires only standard black-box evaluations (no auxiliary models, internal state access, or expensive sampling-based confidence estimation) to compute RI.
Empirical Validation: Extensive experiments across 16 models (including open-source and proprietary models like Qwen, Llama, Claude, GPT, and Gemini) and 5 datasets (SimpleQA, PreciseWikiQA, FaithEval, etc.).
Insight into Model Reliability: The study reveals that high factual accuracy does not guarantee reliable refusal behavior, identifying a persistent gap in current LLM capabilities.

Experimental Results

Stability: RI remains stable across different refusal rates induced by varying system prompts. In contrast, heuristic metrics (e.g., F-score, Weighted Score) fluctuate significantly (up to 70% variation) when refusal rates change, failing to capture a consistent underlying ability.
Correlation with Calibration: RI shows a strong positive correlation (85.8%) with AUROC scores derived from sampling-based uncertainty estimation ( $P(\text{Answering})$ ), validating its faithfulness as a measure of rank-calibration.
Model Rankings: RI provides consistent model rankings independent of accuracy and refusal rates. When the monotonic effects of accuracy and refusal rates are removed, heuristic metrics degrade to near-random stability, whereas RI retains high ranking consistency.
Model Family Influence: The strongest predictor of knowledge-aware refusal ability is the model family (training pipeline and data), rather than model scale or general accuracy. For instance, Claude and Qwen models consistently outperform others in RI, while Gemini and GPT-4.1 variants often fall below the regression line.
Context Sensitivity: Models exhibit degraded refusal performance when ground truth is unavailable in the provided context (e.g., in "Unanswerable" or "Inconsistent" scenarios), suggesting an over-reliance on contextual cues rather than internal knowledge calibration.

Significance and Claims

The paper claims that RI captures an essential, previously overlooked dimension of model reliability. While current factuality evaluations focus on accuracy or hallucination rates, RI highlights that even models with high accuracy can possess unreliable and fragile refusal behaviors.

The authors argue that:

Prompting is insufficient: Simply instructing models to be "more cautious" increases refusal rates but does not improve the quality of refusal decisions (RI remains stable and far from perfect).
Need for new evaluation standards: Comprehensive factuality assessment must incorporate knowledge-aware refusal measures. Relying solely on accuracy or standard calibration metrics may mask significant risks in model deployment.
Practical Utility: The two-pass RI evaluation offers a practical, low-cost alternative to expensive calibration methods, enabling more reliable comparisons of model self-knowledge without requiring white-box access or auxiliary training.

The work concludes that while LLMs are improving in factual accuracy, their ability to accurately judge the limits of their own knowledge remains a critical challenge that requires dedicated measurement and improvement.