💻 computer science

Diagnosing and Repairing Factual Errors in RAG under Budget Constraints

This paper introduces D2R-RAG, a model-agnostic and resource-aware framework that enhances the reliability of Retrieval-Augmented Generation under budget constraints by diagnosing failures through interpretable signatures and adaptively selecting corrective actions within explicit latency and VRAM limits.

Original authors: Soroush Hashemifar, Havva Alizadeh Noughabi, Fattane Zarrinkalam, Ali Dehghantanha

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Soroush Hashemifar, Havva Alizadeh Noughabi, Fattane Zarrinkalam, Ali Dehghantanha

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, but slightly forgetful, assistant (the AI) who is trying to answer your questions. To help them, you give them a stack of reference books (the "Retrieval" part). This system is called RAG (Retrieval-Augmented Generation).

The problem is, sometimes the assistant gets it wrong. Why?

The books were wrong: They didn't find the right page in the stack.
The assistant misread the books: They found the right page but wrote a story that didn't match what was actually written.

In the real world, you can't just throw infinite money or time at fixing these mistakes. You have a budget (limited time and computer memory). If you try to fix every mistake by re-reading the whole library or asking a super-expensive expert, you run out of money or time before you get an answer.

This paper introduces D2R-RAG, a new "fix-it" system designed to work within these tight budgets. Here is how it works, using simple analogies:

1. The Detective (Diagnosis)

Before trying to fix the answer, D2R-RAG acts like a quick detective. It looks at three things:

The question you asked.
The pages the assistant found.
The answer the assistant wrote.

It asks two simple questions:

"Did the assistant find the right evidence?" (If not, the problem is the search).
"Did the assistant write the truth based on that evidence?" (If not, the problem is the writing).

It creates a "failure signature," which is like a specific code telling the system exactly what went wrong, without needing to peek inside the assistant's brain (which is often a "black box" we can't see into).

2. The Smart Shopper (Adaptive Repair)

Once the detective identifies the problem, the system has to choose how to fix it. It has a menu of options, but each costs something different:

Rewrite the question: Make it clearer (Cheap, fast).
Re-read the books: Look at more pages or different types of books (Medium cost).
Call a super-expert: Use a heavy-duty tool to check the facts (Expensive, slow).

The system uses a strategy called a "Contextual Bandit." Think of this like a smart shopper who has a limited amount of cash.

If the shopper knows the store usually has the item on the top shelf, they grab it quickly (low cost).
If they know the item is usually in the back, they spend a bit more time walking there.
They learn from every trip: "Hey, I spent too much time on that last one; next time, I'll try a different aisle first."

The goal is to pick the cheapest fix that actually works, rather than just throwing the most expensive tool at every problem.

3. The Results (What they found)

The authors tested this on two types of tasks: checking facts (like verifying news headlines) and answering complex questions that require connecting multiple dots.

Better Accuracy: D2R-RAG got more answers right than systems that just guess or use a "one-size-fits-all" approach.
Better Efficiency: It didn't waste money. When the problem was just a bad search, it didn't pay for a super-expensive re-write. It fixed the search instead.
Budget Control: Even when they forced the system to be very strict with its time and memory limits, it still managed to find good answers, whereas other systems either crashed or gave up.

The Bottom Line

Think of D2R-RAG as a mechanic for AI. Instead of replacing the whole engine (which is expensive and requires fine-tuning) every time a car makes a weird noise, this mechanic listens to the noise, figures out if it's a loose belt or a bad spark plug, and uses the smallest, cheapest tool needed to fix it. This keeps the car running smoothly without draining your wallet.

The paper concludes that this "diagnose-then-repair" approach is the best way to make AI reliable when you are working with limited resources and cannot see inside the AI's internal code.

Technical Summary: Diagnosing and Repairing Factual Errors in RAG under Budget Constraints

Problem Statement

Retrieval-Augmented Generation (RAG) aims to improve the factuality of Large Language Models (LLMs) by grounding responses in external evidence. However, real-world deployments remain fragile, often failing due to missing or weakly relevant evidence, or generation that does not faithfully reflect the retrieved context. These failure modes are particularly problematic in settings with strict latency and hardware budgets (e.g., limited GPU memory, rate-limited APIs, or cost-constrained edge deployments), where iterative retrieval or resource-intensive reranking are computationally prohibitive.

Existing solutions face significant limitations:

Training-time modifications (e.g., Self-RAG) require fine-tuning and architectural coupling, limiting use in black-box or API-based settings.
Inference-time control (e.g., DRAGIN) often relies on internal model signals like logit access, which are unavailable in closed-source LLM APIs.
Learning-based policies (e.g., MBA-RAG, QueryBandits) often target specific pipeline stages or require additional supervision and modeling components.

There is a need for a model-agnostic, resource-aware framework that can diagnose failures using only observable artifacts and select the least-cost corrective action under explicit latency and VRAM constraints.

Methodology: D2R-RAG

The authors propose D2R-RAG (Diagnose-to-Repair RAG), a framework that augments a standard RAG pipeline with two primary modules: Failure Diagnosis and Adaptive Repair.

1. Failure Diagnosis

This stage determines the trustworthiness of the initial RAG output and identifies the failure type using only observable signals: the query ( $q$ ), retrieved evidence ( $E$ ), generated response ( $r$ ), and a predicted label ( $y$ ). The system triangulates verification using three complementary signals:

Query Entailment ( $e_q$ ): Checks if evidence entails the query.
Response Entailment ( $e_r$ ): Checks if evidence entails the response.
Knowledge Graph (KG) Alignment ( $\kappa$ ): Extracts relational triples from the response and aligns them against a KG to identify entity/predicate mismatches.

Based on these signals, the system classifies failures into four interpretable signatures:

Wrong-Predicate (WP): KG conflict.
Insufficient-Evidence (IE): Retrieved evidence does not entail the query ( $e_q = \text{neutral}$ ).
Wrong-Response (WR): Response is not entailed by evidence ( $e_r \in \{\text{neutral, contradict}\}$ ).
Label–Evidence Mismatch (LEM): Specifically for FEVER, where the predicted label conflicts with query-context entailment.

For FEVER tasks, IE and LEM cases trigger an "Unverified" abstention label to prioritize retrieval-oriented repairs.

2. Adaptive Repair via Contextual Bandits

Once a failure signature is identified, D2R-RAG formulates repair selection as a Contextual Multi-Armed Bandit (CMAB) problem.

Action Space ( $A$ ): A small, black-box deployable set of interventions:
1. Prompt-level rewrites: Paraphrasing or simplifying the query to reduce ambiguity.
2. Reranker activation: Using a cross-encoder to improve evidence precision.
3. Retrieval-level interventions: Adjusting depth ( $k$ ), switching between BM25 and dense retrieval, or refreshing the index.
Policy Learning: The system uses LinUCB to learn a policy that balances expected factual recovery against computational cost. The policy assumes a linear reward model $\hat{r}(a | x) = x^\top \theta_a$ and selects actions by maximizing an upper-confidence bound.
Resource-Aware Reward: The reward function ( $r(a)$ $r (a)$ ) scores repairs based on output reliability (NoFailure, KG consistency, NLI entailment) and budget compliance. It includes:
- Hard gates: Zeroing out rewards if latency ( $L_a$ ) or VRAM ( $V_a$ ) exceeds per-action budgets ( $B_L, B_V$ ).
- Multiplicative terms: Encouraging cheaper actions among feasible ones.

Key Contributions

Model-Agnostic Framework: D2R-RAG operates without generator fine-tuning or access to internal logits, enabling deployment with closed-source LLM APIs.
Interpretable Failure Signatures: The system distinguishes between retrieval-side insufficiency and generation-side unfaithfulness using only observable artifacts.
Adaptive, Budget-Constrained Repair: By framing repair as a contextual bandit problem with explicit latency and VRAM constraints, the system learns to select the "least-cost" corrective action for specific failure modes.
Empirical Validation: The framework is evaluated on fact verification (FEVER) and multi-hop question answering (HotpotQA), demonstrating improved reliability and accuracy-efficiency trade-offs compared to recent baselines.

Experimental Results

The authors evaluated D2R-RAG against Naive-RAG, Query Paraphrase, Context Expansion, and Thompson Sampling (a context-free policy baseline) using GPT-4o-mini.

FEVER (Fact Verification):
- D2R-RAG (Thompson Sampling) achieved 61.5% Accuracy, outperforming Naive-RAG (56.3%) and Context Expansion (61.8% with higher latency).
- It maintained a superior cost profile (1.14s latency, 0.17 MB VRAM) compared to the LinUCB variant (1.47s, 0.36 MB).
HotpotQA (Multi-hop QA):
- D2R-RAG (Thompson Sampling) reached 40.4% Exact Match (EM) with 71.52% faithfulness.
- The LinUCB variant matched 39.8% EM but incurred higher latency (3.41s vs 2.37s) and occasionally exceeded VRAM budgets.
Policy Adaptation (RQ2): Analysis showed the policy correctly conditions actions on failure types. For example, "Insufficient-Evidence" cases on FEVER favored retrieval escalation (deeper dense/BM25), while "Wrong-Response" and "Wrong-Predicate" failures on HotpotQA routed to reranker activation.
Budget Sensitivity (RQ3): Ablation studies confirmed that the reward formulation effectively controls quality-cost trade-offs. "Stringent" budget settings reduced VRAM usage while preserving performance, while "Relaxed" settings lowered latency and boosted faithfulness.

Significance and Limitations

The paper argues that reliable recovery in black-box RAG settings requires a dual capability: lightweight diagnostic differentiation of failure types and the selection of the least-cost corrective action under resource constraints. D2R-RAG demonstrates that reliability improves most when repair is targeted to specific failure signatures rather than using a uniform strategy.

The authors modestly note two limitations:

Diagnostic Signal Imperfections: KG alignment and NLI models may struggle with noisy evidence and multi-hop reasoning, potentially cascading errors into suboptimal repairs.
Action Space Constraints: The policy is limited to the exposed repair actions; if cheaper alternatives are unavailable, the system may overuse costly patches or lack the capability to resolve complex failures.

The work concludes that D2R-RAG offers a practical, resource-aware approach to enhancing RAG reliability in constrained environments without requiring access to internal model parameters.