💻 computer science

Dynamic Knowledge Grounding for Retrieval-Augmented Generation: Uncertainty-Aware Evidence Fusion with Adaptive Retrieval Control

FORGE is a principled framework that enhances Retrieval-Augmented Generation by integrating an uncertainty-driven adaptive retrieval controller, a reliability-weighted evidence fusion network, and a token-level grounding verification module to significantly reduce hallucinations and computational costs while achieving state-of-the-art performance across multiple benchmarks.

Original authors: Babasaheb Satpute, Chaya Jadhav, Rajesh Bharati, Deepak Kshirsagar

Published 2026-11-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Babasaheb Satpute, Chaya Jadhav, Rajesh Bharati, Deepak Kshirsagar

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of artificial intelligence, large language models have emerged as powerful engines of text, capable of writing stories, solving problems, and answering questions with a fluency that often mimics human thought. These systems work by drawing on a massive internal library of facts and patterns they absorbed during training. However, this internal library has a significant flaw: it is static and can become outdated, and more dangerously, the model can confidently invent facts that never happened, a phenomenon known as "hallucination." To fix this, researchers developed a method called retrieval-augmented generation. This approach acts like a librarian for the AI, allowing it to pause its writing and look up answers in an external database of documents before speaking. While this helps, current systems often behave clumsily, fetching information for every single question regardless of whether it is needed, and then blending that information in a simple, unthinking way that can still lead to errors.

A team of researchers has now introduced a new framework called FORGE, designed to make this process smarter, more efficient, and far more reliable. Instead of treating every question the same, FORGE acts like a careful editor that decides exactly when to look up information, how to weigh the reliability of different sources, and how to verify that the final answer is actually supported by what it found. The system is built around three distinct parts that work together seamlessly. First, it learns to recognize when it is unsure about an answer and only then reaches for its external knowledge, saving time and avoiding the noise of irrelevant data. Second, when it does gather information, it does not just dump the text into the answer; instead, it carefully analyzes the quality of each document, checking if different sources agree with one another and giving more importance to the most trustworthy passages. Finally, as the system generates its response, it constantly checks every single word it writes to ensure it is backed up by the evidence it found, refusing to make claims that lack support.

The researchers tested this new approach across eight different challenging benchmarks, ranging from simple fact-checking to complex questions that require connecting ideas from multiple documents. The results showed that FORGE significantly outperformed existing methods. It improved the accuracy of answers by between 2.1 and 4.8 points compared to the best previous systems, while simultaneously cutting the rate of hallucinations by nearly 39 percent. Perhaps just as importantly, the system became much more efficient. By learning to skip the lookup step for questions it already knew well, it reduced the number of times it needed to search for information by more than 42 percent. This means the system is not only more truthful but also faster and less computationally expensive to run.

The success of FORGE relies on a shift in how the AI interacts with its knowledge base. Rather than blindly retrieving information for every query, the system uses a learned policy to gauge its own uncertainty. If the model feels confident, it answers from memory; if it senses a gap in its knowledge, it triggers a search. When it does search, it employs a sophisticated fusion network that treats different documents differently, assigning higher weight to sources that are authoritative and consistent with one another. This prevents the system from being misled by a single unreliable document. Furthermore, a verification module runs in the background, assigning a confidence score to every word generated. If the system cannot find evidence to support a specific claim, it can choose to hold back rather than invent a fact. This granular level of control allows the system to maintain a high standard of truthfulness without sacrificing the flow of natural language.

In their analysis, the researchers demonstrated that these improvements are not just lucky accidents but are grounded in rigorous mathematical principles that guarantee the system behaves optimally over time. They showed that the method for deciding when to retrieve information converges toward the best possible strategy, minimizing wasted effort. They also proved that the way information is combined is theoretically optimal for preserving the most useful details while discarding the rest. Most critically, they established that the verification process provides a reliable measure of confidence, ensuring that the system's claims are consistently tied to real evidence. The study confirms that by integrating these three capabilities—adaptive searching, intelligent blending, and strict verification—into a single framework, it is possible to build artificial intelligence that is not only more knowledgeable but also more honest and efficient. This work suggests a clear path forward for creating AI tools that users can trust with high-stakes information, from medical advice to legal research, where accuracy and the ability to cite sources are essential.

Technical Summary: Dynamic Knowledge Grounding for Retrieval-Augmented Generation

1. Problem Statement

While Retrieval-Augmented Generation (RAG) has emerged as a paradigm for grounding Large Language Model (LLM) outputs in external knowledge, current approaches suffer from three critical limitations:

Static Retrieval: Existing systems retrieve a uniform number of passages for every query, regardless of complexity or the model's existing parametric knowledge. This leads to wasted computation and the introduction of irrelevant noise for queries the model could answer confidently from memory.
Simple Evidence Fusion: Standard RAG systems typically combine retrieved passages with the query using shallow concatenation or simple attention. This approach fails to model differential relevance, source reliability, or inter-passage consistency, treating all evidence as equally trustworthy.
Unverified Grounding: Despite retrieval augmentation, LLMs can still generate content that conflicts with retrieved evidence or makes unsupported claims. Current methods lack explicit, token-level verification mechanisms to prevent the propagation of hallucinations in high-stakes scenarios.

2. Methodology: The FORGE Framework

The authors propose FORGE (Factually-Optimized Retrieval for Grounded gEneration), a unified framework designed to address these limitations through three tightly integrated modules:

A. Adaptive Retrieval Controller (ARC)

The ARC module frames adaptive retrieval as an online learning problem to determine when, what, and how much to retrieve.

Uncertainty-Driven Triggering: At each generation step, ARC estimates the model's epistemic uncertainty (reducible by retrieval) versus aleatoric uncertainty (irreducible). Retrieval is triggered only when epistemic uncertainty exceeds a learned threshold.
Strategy Selection: When retrieval is triggered, ARC employs a contextual multi-armed bandit formulation with Upper Confidence Bound (UCB) exploration. It selects among strategies such as single-hop, multi-hop, query decomposition, or skipping retrieval based on the current context (uncertainty, hidden state, query embedding).
Theoretical Basis: The policy is designed to minimize retrieval regret, ensuring sublinear regret bounds relative to the optimal fixed policy.

B. Evidence Fusion Network (EFN)

The EFN performs multi-granularity fusion of retrieved evidence, moving beyond simple concatenation.

Multi-Granularity Attention: Passages are encoded at three levels: token, sentence, and passage. The network computes cross-attention at each granularity level to capture different scopes of information.
Reliability Weighting: Each passage is assigned a learned reliability score ( $r_j$ $r_{j}$ ) based on three factors:
1. Relevance: Query-passage similarity.
2. Source Authority: The inherent quality of the source.
3. Consistency: Inter-passage agreement (measured via Natural Language Inference scores).
Gated Fusion: A learned gate dynamically combines the multi-granularity representations, weighted by reliability, to produce an optimal fused context. This is theoretically grounded in the Information Bottleneck and rate-distortion frameworks.

C. Grounding Verification Module (GVM)

The GVM provides fine-grained, token-level factual consistency checking.

Token-Level Scoring: For every generated token, the GVM calculates a grounding score representing the entailment of that token by the retrieved evidence cache.
Calibration: Raw scores are calibrated using temperature scaling and Platt calibration to ensure that the confidence scores are statistically reliable (i.e., a score of 0.8 implies an 80% probability of being grounded).
Selective Abstention: If the cumulative grounding confidence falls below a threshold, the system can abstain from generating further content rather than producing unfaithful output.
Decoding Bias: The module modifies the next-token distribution to bias generation toward well-supported tokens.

Training Procedure

FORGE is trained in three stages:

Pre-training: Joint training of the retriever and EFN using contrastive loss on passage relevance.
Fine-tuning: Generator fine-tuning with active EFN, optimizing for task performance and penalizing low grounding scores.
Joint Optimization: Reinforcement learning (REINFORCE) to jointly optimize the retrieval policy, fusion, and verification modules, balancing task reward against retrieval costs.

3. Key Contributions

The paper claims four primary contributions:

Unified Framework: A single architecture (FORGE) that jointly optimizes adaptive retrieval, evidence fusion, and grounding verification.
Rigorous Theoretical Analysis:
- Retrieval Regret: Proven $O(\sqrt{T} \log K)$ regret bounds for the adaptive retrieval policy.
- Fusion Optimality: Information-theoretic optimality guarantees for evidence fusion under the rate-distortion framework.
- Generalization Bounds: PAC-Bayes bounds for the grounding verification module.
- Hallucination Reduction: End-to-end guarantees on hallucination rate reduction.
State-of-the-Art Performance: Empirical results showing significant improvements over baselines across eight diverse benchmarks.
Comprehensive Analysis: Ablation studies demonstrating the complementary nature of each module and the interpretability of the learned retrieval policies.

4. Experimental Results

The framework was evaluated on eight benchmarks: Natural Questions, TriviaQA, HotpotQA, FEVER, ASQA, ELI5, PopQA, and TruthfulQA.

Performance Gains: FORGE achieved state-of-the-art results, improving performance by 2.1–4.8 points over the best baselines (e.g., Self-RAG, CRAG) on average.
Hallucination Reduction: The system reduced hallucination rates by 38.7% relative to the strongest baseline (CRAG), achieving a final hallucination rate of 11.9% on TruthfulQA (compared to CRAG's 18.4%).
Efficiency: By adaptively skipping retrieval for queries the model can answer from memory, FORGE reduced retrieval costs by 42.3% compared to "always-retrieve" baselines, while simultaneously lowering latency.
Ablation Findings:
- Removing ARC (forcing always-retrieve) degraded efficiency but maintained high accuracy, validating the cost-saving nature of the policy.
- Removing EFN (using simple concatenation) caused a significant drop in quality (+4.1 EM points lost), highlighting the importance of reliability-weighted fusion.
- Removing GVM increased hallucination rates by 36.4%, confirming its role in factual consistency.

5. Significance and Claims

The authors position FORGE as a principled advancement in RAG systems that moves beyond heuristic designs to theoretically grounded mechanisms.

Trustworthiness: By providing calibrated, token-level grounding scores and attribution, FORGE addresses the "black box" nature of RAG, making it suitable for high-stakes applications where provenance and accuracy are critical.
Efficiency-Effectiveness Trade-off: The framework demonstrates that adaptive retrieval does not require sacrificing accuracy; rather, it improves it by reducing noise and focusing computational resources on queries that genuinely require external knowledge.
Limitations: The authors modestly acknowledge limitations, including computational overhead from multi-granularity fusion, dependence on corpus coverage (struggling with very recent events or niche topics not in the index), and vulnerability to corpus poisoning if all retrieved evidence is uniformly misleading. They suggest future work in multi-modal fusion, personalized policies, and formal verification.

In conclusion, FORGE offers a systematic approach to dynamic knowledge grounding, combining uncertainty quantification, information-theoretic fusion, and calibrated verification to produce more reliable, efficient, and truthful generative AI.