💬 NLP

EvidenceRL: Reinforcing Evidence Consistency for Trustworthy Language Models

The paper introduces EvidenceRL, a reinforcement learning framework using Group Relative Policy Optimization to enforce evidence adherence in Large Language Models, which significantly reduces hallucinations and improves grounding and faithfulness in high-stakes domains like cardiac diagnosis and legal reasoning without compromising task accuracy.

Original authors: J. Ben Tamo, Yuxing Lu, Benoit L. Marteau, Micky C. Nnamdi, May D. Wang

Published 2026-03-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: J. Ben Tamo, Yuxing Lu, Benoit L. Marteau, Micky C. Nnamdi, May D. Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, well-read student named LLM (Large Language Model). This student can write beautiful essays, solve complex problems, and sound incredibly confident. However, there's a catch: LLM loves to make things up.

If you ask LLM for medical advice or legal reasoning, it might give you a perfect-sounding answer that is actually a complete fabrication. It's like a student who memorized the style of a textbook but forgot to actually read the facts. In high-stakes fields like healthcare or law, this "hallucination" is dangerous because a wrong answer can hurt someone.

The paper you shared introduces a new training method called EvidenceRL. Think of it as a strict, super-observant Coach who forces the student to stop guessing and start proving their work.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Confident Liar"

Currently, if you ask an AI to diagnose a heart condition, it might say, "The patient has X," and then invent a fake reason why. Even if you give the AI a stack of real medical records (evidence), the AI often ignores them and relies on what it "remembers" from its training data. It's like a lawyer arguing a case by making up facts because they sound good, rather than looking at the actual police report.

2. The Solution: EvidenceRL (The "Proof-First" Coach)

The researchers created a new training system called EvidenceRL. Instead of just telling the AI, "You got the right answer," they changed the rules of the game. Now, the AI only gets a "good grade" if it can prove its answer using the specific documents provided.

They used a technique called Reinforcement Learning. Imagine a dog training session:

Old Way: The dog sits, and you give it a treat because it looks cute.
EvidenceRL Way: The dog only gets a treat if it sits and brings you the specific ball you asked for. If it sits but brings a shoe, no treat.

3. How the Coach Checks the Work (The "Focus-Then-Verify" Trick)

Checking if an AI is telling the truth is hard. If you ask, "Is this whole essay true?" it's too vague. The AI might get one sentence right and ten wrong, and the score gets diluted.

The researchers invented a clever trick called Focus-Then-Verify:

The Metaphor: Imagine a detective looking at a massive crime scene. Instead of trying to verify the whole scene at once, the detective picks one specific clue (like a muddy footprint) and asks, "Does this specific clue support the suspect's story?"
The Tech: The system breaks the AI's answer down sentence by sentence. It takes one sentence of the answer and one piece of the evidence (like a medical report) and asks a specialized "Judge" (a smaller AI): "Does this evidence prove this sentence?"
If the evidence supports it, the AI gets points. If the AI makes up a fact that contradicts the evidence, it gets penalized heavily.

4. The Results: From "Lucky Guess" to "Evidence-Based"

The paper tested this on two very serious jobs: Cardiac Diagnosis (heart doctors) and Legal Reasoning (lawyers).

Before EvidenceRL: The AI was like a gambler. It might get the right diagnosis (the "lucky guess"), but it couldn't explain why using the patient's actual records. It was often just guessing based on patterns it had seen before.
After EvidenceRL: The AI became a detective.
- Fewer Hallucinations: The number of made-up facts dropped by nearly 5 times.
- More Proof: The number of answers actually supported by the evidence jumped from about 32% to 61%.
- Better Accuracy: Surprisingly, by forcing the AI to stick to the facts, it actually got more correct answers, not fewer.

5. Why This Matters

The most important takeaway is that you can't just "filter" the bad answers after the AI generates them. You have to change how the AI thinks while it's learning.

Inference-time controls (like asking the AI to "think harder" or "check its work" after it writes) are like telling a liar to "try to be honest" after they've already told the lie. It doesn't work well.
EvidenceRL changes the AI's brain during training. It teaches the AI that truth is only valid if it has a receipt.

Summary Analogy

Think of the AI as a tour guide.

Old AI: The guide knows the city well but often makes up stories about landmarks to sound interesting. Tourists get confused and lost.
EvidenceRL AI: The guide is now required to carry a map and point to the exact spot on the map for every story they tell. If they can't point to the spot, they aren't allowed to tell the story.

The result? The tour is still fun and informative, but now it's trustworthy. In fields like medicine and law, that trust is the difference between a good decision and a disaster.

1. Problem Statement

Large Language Models (LLMs) are fluent but prone to hallucinations, particularly in high-stakes domains like healthcare and law. A critical failure mode is the lack of evidence grounding: models often generate plausible, factually correct answers based on parametric memory (internal knowledge) rather than the provided context, or they produce answers that contradict the available evidence.

Existing solutions have limitations:

Retrieval-Augmented Generation (RAG): While RAG provides context, models often ignore it or engage in "post-rationalization" (attaching superficial citations to answers derived from memory).
Inference-time Controls: Methods like Self-RAG or Self-Consistency filter outputs but do not fundamentally alter the model's generation policy or reasoning behavior.
Post-hoc Verification: Checking facts after generation does not prevent the model from learning to ignore evidence during training.
Standard RLHF: Human feedback often rewards plausibility over verifiability, failing to explicitly optimize for evidence adherence.

The core challenge is that current systems lack a mechanism to directly enforce evidence-consistent generation during the training process, leading to a "faithfulness gap" where models are accurate but not grounded.

2. Methodology: EvidenceRL

The authors propose EvidenceRL, a reinforcement learning framework that optimizes evidence adherence as a differentiable training objective. It utilizes Group Relative Policy Optimization (GRPO) to train models without relying on human preference annotations.

Key Components:

A. Reward Architecture
The training reward ( $r$ ) is a weighted sum of three components:

Format Reward ( $r_f$ ): A binary signal ensuring the output is valid JSON, facilitating structured parsing.
Correctness Reward ( $r_c$ ):
- Medical: Uses a domain-specific embedding (BioLORD-2023) to compute cosine similarity between predicted diagnoses and ground truth.
- Legal: Uses exact matching for multiple-choice answers.
Grounding Reward ( $r_g$ ): The core innovation, computed via a "Focus-Then-Verify" strategy to avoid signal dilution in long contexts.
- Focus: An anchor context (e.g., chief complaint) is combined with individual evidence snippets (e.g., specific lab results or legal statutes) to create focused premise-hypothesis pairs.
- Verify: A frozen Natural Language Inference (NLI) model (e.g., PubMedBERT for medical, DeBERTa for legal) scores the entailment between the model's reasoning and each evidence snippet.
- Aggregation: The final grounding score is derived from the strongest supporting or contradicting evidence ( $r_{max}^g$ ) and the average consistency ( $r_{avg}^g$ ).

B. Optimization (GRPO)

For each input, the policy samples a group of $G$ completions.
Rewards are normalized within the group to calculate advantages.
The policy is updated using a clipped objective with KL regularization against a frozen reference policy, encouraging the model to generate outputs that are simultaneously correct and evidence-grounded.

3. Key Contributions

EvidenceRL Framework: A novel RL framework that enforces evidence grounding as a primary training objective using automated NLI and LLM-judge rewards, eliminating the need for human preference data.
Focus-Then-Verify Architecture: A mechanism to compute sentence-level entailment within focused context windows, resolving the signal dilution issues inherent in document-level scoring.
Empirical Validation: Demonstration that EvidenceRL improves both task accuracy and evidential grounding across multiple model families (Llama, Gemma, GPT-oss) and scales (3B–120B) in high-stakes medical and legal domains.

4. Experimental Results

The framework was evaluated on two datasets: MIMIC-IV-Ext (Cardiac Diagnosis) and BarExam MBE (Legal Reasoning).

Medical Domain (Cardiac Diagnosis)

Performance: On Llama-3.2-3B, EvidenceRL increased F1@3 from 37.0 to 54.5 and Grounding (Gmax@3) from 47.6 to 78.2.
Hallucinations: Hallucination rates dropped nearly 5x (from 11.6% to 2.4%).
Evidence-Based Rate: The percentage of diagnoses supported by evidence increased from 31.8% to 61.6%.
Faithfulness: The fraction of correct predictions that were genuinely evidence-supported rose to 87.5%.

Legal Domain (BarExam)

Performance: On Llama-3.1-8B, Faithfulness increased from 32.8% to 67.6%.
Grounding: Average grounding scores improved significantly (from -1.7 to 18.9), shifting predictions from "Lucky Guesses" to "Evidence-Based" reasoning.

Comparative Analysis

vs. SFT: Supervised Fine-Tuning (SFT) improved accuracy but caused a "grounding collapse" (models learned to cite evidence format without semantic adherence).
vs. Inference-time Controls: Methods like Self-RAG and Self-Consistency failed to consistently improve both accuracy and grounding simultaneously.
vs. fDPO: Faithfulness DPO (fDPO) improved grounding but often at the cost of accuracy or by distilling knowledge from stronger models. EvidenceRL improved both accuracy and grounding by optimizing on-policy.
Scaling: EvidenceRL allowed smaller models (e.g., 3B) to outperform larger zero-shot baselines (e.g., 70B) in terms of evidence grounding, suggesting that training methodology can partially substitute for parameter scale.

5. Significance and Conclusion

EvidenceRL addresses a fundamental limitation of current LLMs: the ability to produce correct answers that are not derived from provided evidence. By shifting the learning objective to explicitly penalize unsupported correctness, the framework reshapes the model's decision policy.

Trustworthiness: It provides a path to deploy LLMs in safety-critical domains where decisions must be justifiable by verifiable information.
Efficiency: It offers a compute-efficient path to reliable reasoning, as smaller models trained with EvidenceRL can outperform larger, untrained models in grounding.
Generalizability: The approach is domain-agnostic, demonstrated successfully in both clinical and legal reasoning.

The paper concludes that inference-time controls (like retrieval) are insufficient for changing how models reason; only training-time reinforcement of evidence consistency can fundamentally align model behavior with the requirement for verifiable, evidence-based outputs.