🤖 AI

Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching

This paper demonstrates that a HOPE-inspired, memory-augmented multi-agent pipeline utilizing semantic caching and progressive review stages can significantly reduce LLM hallucinations, improve operational efficiency by cutting energy consumption, and enhance auditability without requiring model retraining.

Original authors: Diego Gosmar, Deborah A. Dahl

Published 2026-05-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Diego Gosmar, Deborah A. Dahl

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a high-stakes newsroom where a story needs to be written, fact-checked, and edited before it goes to print. The problem is that your first writer is a brilliant but chaotic artist who loves to make things up just to sound interesting. If you let them publish, the whole newspaper becomes a collection of lies.

This paper describes a system designed to stop those lies from spreading, using a team of AI agents, a "smart memory" system, and a new way of grading their work.

Here is how the system works, broken down into simple parts:

1. The Chaotic Writer (The FrontEndAgent)

The system starts with a specific design choice: the first AI agent is deliberately set to be unreliable. Think of this agent as a writer with a temperature setting of 1.0 (the highest possible creativity). They are told to answer every question with total confidence, even if they have to invent details. They are not allowed to say "I'm not sure" or "this might be made up."

Why do this? The researchers wanted to create a worst-case scenario. By forcing the first agent to hallucinate (make things up) constantly, they could clearly measure how well the rest of the team fixes those mistakes.

2. The Two-Step Correction Team

Once the chaotic writer produces a story, it doesn't go straight to the reader. It passes through two other agents:

The Second-Level Reviewer (The Editor): This agent is more cautious. It reads the chaotic story, finds the made-up facts, and rewrites them to be safer. It adds notes like "this is a guess" or "we don't have proof." It acts like a strict editor who refuses to let unverified claims slide.
The Third-Level Reviewer (The Final Polish): This agent takes the edited story and makes it look clean and professional for the final reader. It removes the messy internal notes but ensures the final text is accurate and doesn't sound like a lie.

The Result: The paper found that this "assembly line" successfully reduced the amount of hallucination by about 31% to 36%. The more cautious the final output, the better the score.

3. The "Smart Filing Cabinet" (Semantic Caching)

Running three different AI agents for every single question is expensive and slow. It's like hiring three people to write three drafts for every single email you send.

To fix this, the system uses a Continuum Memory System (CMS). Imagine a smart filing cabinet that doesn't just look for exact matches (like searching for the word "apple"), but understands meaning.

If you ask, "What is the capital of France?" and the system just answered "Paris" five minutes ago, it remembers that.
If you ask, "Tell me about the capital city of France," the system recognizes these are the same question and pulls the answer from the cabinet instead of asking the AI to think again.

The Benefit: This system saved them from asking the AI to think 47% of the time. This means less energy used, less carbon dioxide emitted, and faster responses, making it possible to run this three-person team in the real world without breaking the bank.

4. The New Scorecard (The KPIs and THS)

How do you know if the system is working? The researchers created a new report card called the Total Hallucination Score (THS).

Instead of just counting lies, they look at five things:

Factual Claim Density: How many unverified facts are there? (Lower is better).
Factual Grounding: Did they cite real sources? (Higher is better).
Disclaimer Frequency: Did they admit when they were guessing? (Higher is better).
Contextualization: Did they frame the answer correctly? (Higher is better).
Observability: Can we see how the AI thought? (Higher is better).

They tested five different ways of weighing these scores. They found that the configuration which prioritized transparency and "showing your work" (Observability) actually resulted in the best reduction of lies. In other words, making the AI more transparent didn't hurt its accuracy; it helped it.

The Big Takeaway

The paper concludes that you don't need to retrain the AI models to make them more honest. Instead, you can:

Set up a team of agents where one is allowed to be creative (and wrong) and the others are strict fact-checkers.
Give them a shared, smart memory so they don't have to re-invent the wheel for every question.
Grade them on how well they admit uncertainty and show their work.

This approach creates a system that is more reliable, cheaper to run, and easier to audit, all without changing the underlying AI brain.

Technical Summary: Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching

Problem Statement

The deployment of Large Language Models (LLMs) in production systems faces a critical reliability gap: the generation of factually unsupported claims (hallucinations) with high confidence. While problematic in single-model settings, this issue is exacerbated in multi-agent pipelines where unchecked hallucinations can propagate and amplify across stages. Existing defensive architectures, primarily designed for prompt-injection attacks, typically operate as single-pass filters (input-side) or judges (output-side) and fail to leverage accumulated context from prior interactions. Furthermore, multi-stage review pipelines often face operational infeasibility at scale due to the latency and cost of sequential LLM invocations, alongside the lack of model retraining required to improve factual grounding.

Methodology

The authors propose a three-stage agentic pipeline orchestrated via the Open Floor Protocol (OFP), integrated with a Continuum Memory System (CMS) inspired by the HOPE (Hierarchical Organization of Processing and Experience) framework's Nested Learning architecture.

1. Agentic Pipeline Architecture

The system employs four distinct agents running on Llama 3.1 via Ollama, differentiated by inference parameters and system prompts:

FrontEndAgent (1st Stage): Deliberately configured as a "weak," high-stochasticity generator ($temp=1.0$). It is instructed to answer with full confidence and avoid disclaimers, creating a measurable baseline of hallucinations (high Factual Claim Density).
SecondLevelReviewer (2nd Stage): Acts as the primary corrector ($temp=0.1$). It detects unsupported claims, replaces them with cautious wording, and outputs structured metadata (utterance, whisper_context, whisper_value).
ThirdLevelReviewer (3rd Stage): Functions as the final factuality enforcer ($temp=0.05$). It produces clean, user-facing text without internal reasoning metadata, prioritizing authoritative framing.
KPI Evaluator (4th Agent): A read-only observer ($temp=0.0$) that computes evaluation metrics independently of the decision path.

2. Continuum Memory Systems (CMS) & Semantic Caching

To address the stateless nature of standard inference engines and enable "Nested Learning," each agent is equipped with a CMS comprising:

Medium-Term Memory (MTM): An LRU (Least Recently Used) cache storing recent prompt-response pairs.
Long-Term Memory (LTM): An LFU (Least Frequently Used) cache for recurring patterns, populated via periodic consolidation from MTM.
Semantic Indexing: Before invoking the LLM, agents embed prompts using all-MiniLM-L6-v2 and query the cache using cosine similarity with a threshold $\tau = 0.87$ . If a semantic match is found, the cached response is reused; otherwise, a fresh inference is performed.

3. Evaluation Framework

The system was evaluated on a hybrid benchmark of 310 prompts:

70% Realistic Epistemic-Uncertainty Prompts: Questions where agents should hedge rather than confabulate.
30% Fabrication-Induction Stress-Test Prompts: Designed to pressure the pipeline into hallucinating on demand.

Performance is measured via five Key Performance Indicators (KPIs) aggregated into a Total Hallucination Score (THS):

FCD (Factual Claim Density): Lower is better.
FGR (Factual Grounding References): Higher is better.
FDF (Fictional Disclaimer Frequency): Higher is better.
ECS (Explicit Contextualization Score): Higher is better.
OSR (Observability Score Ratio): A new metric measuring the transparency of reasoning and metadata.

The THS formula subtracts mitigation signals (FGR, FDF, ECS, OSR) from the risk signal (FCD). A more negative THS indicates stronger mitigation. Five weighting configurations were tested, ranging from a balanced Baseline to ExtremeObservability.

Key Results

1. Mitigation Effectiveness

The asymmetric pipeline design yielded significant end-to-end improvements in factual reliability:

THS Reduction: Across five weighting configurations, the pipeline achieved a Total Hallucination Score reduction of −31.3% to −35.9% from the FrontEndAgent to the ThirdLevelReviewer.
U-Shaped Trajectory: The SecondLevelReviewer achieved the lowest (best) local THS scores (e.g., −0.1005 in ExtremeObservability). The ThirdLevelReviewer showed a slight rebound in THS (e.g., −0.0709) as it traded verbose hedging (high FDF/ECS) for compact, authoritative output, yet remained substantially better than the initial generator.
Observability Impact: The ExtremeObservability configuration (heavily weighting OSR, FDF, and ECS) achieved the most negative final THS (−0.0709), confirming that prioritizing transparency and auditability reinforces rather than compromises mitigation.

2. Efficiency and Sustainability

Semantic caching significantly reduced the computational load, making multi-stage pipelines viable at scale:

Cache Hit Rate: The system achieved 440 cache hits out of 930 potential calls, resulting in a 47.3% hit rate.
Inference Reduction: Effective LLM invocations were reduced from 930 to 490, lowering energy consumption and CO2e footprint without requiring model retraining.
Uniform Distribution: Hit rates were consistent across stages (46.1% → 47.1% → 48.7%), indicating that efficiency gains are distributed rather than concentrated in isolated bursts.

Significance and Claims

The paper claims to demonstrate that memory-augmented multi-agent designs can jointly improve factual reliability, operational efficiency, and auditability without modifying model weights.

Operational Viability: By integrating semantic caching, the authors argue that multi-stage review pipelines, often deemed too costly for production, become operationally feasible.
Architectural Insight: The study validates that a deliberately "weak" first agent (high stochasticity) is necessary to create a measurable hallucination load, allowing the corrective value of downstream reviewers to be quantified.
Observability as Mitigation: The findings suggest that weighting observability (OSR) in evaluation functions does not degrade performance but actively reinforces mitigation quality.
Practical Implementation: The work approximates the theoretical principles of Nested Learning (HOPE) using a practical, caching-based approach compatible with existing inference engines, offering a concrete defense against hallucination propagation in agentic systems.

The authors maintain a modest stance, acknowledging that the benchmark is synthetic and the KPI evaluator is model-based. They identify future work as needing larger real-world corpora, cross-judge robustness checks, and direct power instrumentation for infrastructure-aware sustainability reporting.