🤖 AI

ToE: A Hierarchical and Explainable Claim Verification Framework with Dynamic Multi-source Evidence Retrieval and Aggregation

This paper introduces ToE, a hierarchical and explainable framework that dynamically decomposes claims into argument trees and employs reinforcement learning for multi-source evidence retrieval and aggregation, achieving significant improvements in fact-checking accuracy—particularly against AI-generated misinformation—while providing theoretical guarantees on policy convergence.

Original authors: Zhaoqi Wang, Zijian Zhang, Kun Zheng, Zhen Li, Xin Li, Chunlei Li, Jiamou Liu

Published 2026-06-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhaoqi Wang, Zijian Zhang, Kun Zheng, Zhen Li, Xin Li, Chunlei Li, Jiamou Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but the clues you find are a mix of truth, lies, and cleverly disguised forgeries. In the digital world, this is exactly what happens when Artificial Intelligence (AI) tries to fact-check news. Sometimes, bad actors create fake stories so convincing and well-placed that even smart AI gets tricked into believing them.

The paper introduces a new system called ToE (Tree of Evidence). Think of ToE not as a single detective, but as a hierarchical investigation team that builds a "family tree" of facts to solve a claim.

Here is how it works, broken down into simple steps:

1. The Problem: The "Fake News" Trap

Standard AI models are like students who only study from old textbooks. They don't know what happened today. To fix this, we usually let them "Google" things. But here's the catch: bad actors have learned how to "poison" search results. They write fake articles that look so real and rank so high in search engines that the AI grabs them as truth.

The Analogy: Imagine a student trying to write a history essay. They go to the library, but someone has secretly replaced the real history books with fake ones that look identical. The student reads the fake books and writes a wrong essay, thinking they are right.

2. The Solution: Building a "Fact Tree"

Instead of asking one question and getting one answer, ToE treats a claim like a tree trunk that needs to be broken down into smaller branches.

The Claim (The Trunk): "Tim Cook is the CEO of OpenAI."
The Branches (Sub-claims): To verify this, ToE doesn't just ask "Is this true?" It breaks it down:
- Who is the CEO?
- When did they join?
- Where is the headquarters?
- Why would they join?

If the evidence for one branch is weak, ToE grows a new, smaller branch to dig deeper. It keeps expanding the tree until it has enough solid wood (evidence) to make a decision.

3. The Three Specialized Agents

ToE uses three different "agents" (AI workers) that work together:

The Hunter (Retrieval Agent): This agent is like a scout with a map. It doesn't just search one place; it knows that different types of lies need different tools.
- If the claim is about science, it hunts in academic journals.
- If it's about politics, it checks fact-checking websites.
- If it's about a celebrity, it looks at social media.
- Crucially: It is trained to actively look for counter-evidence (proof that the claim is false), not just proof that it's true. This prevents it from falling into "confirmation bias" (only finding what it wants to find).
The Judge (Evaluation Agent): Once the Hunter brings back a pile of documents, the Judge reads them. It asks: "Is this source trustworthy? Does this document actually support the claim or contradict it?" It assigns a score to every piece of evidence.
The Foreman (Aggregation Agent): This agent looks at the whole tree. It takes the scores from all the branches and combines them. If the branches are shaky, it tells the team to go dig deeper. If the branches are solid and agree with each other, it stops the search and declares a verdict.

4. Why It's Smarter Than Others

Most fact-checking systems are like a person reading one newspaper and deciding if it's true. ToE is like a team of investigators cross-referencing five different sources, checking for contradictions, and only stopping when they are 100% sure.

The paper tested ToE against other methods using a special "adversarial" dataset (a set of fake news designed specifically to trick AI).

The Result: While other systems got confused by the fake news, ToE remained steady. It improved accuracy by 4% to 24% compared to the best existing methods.
The "Poison" Test: Even when the fake news was perfectly crafted to look real (a technique called "GEO poisoning"), ToE was able to spot the inconsistencies by digging deeper into its evidence tree.

5. The Bottom Line

ToE is a framework that turns fact-checking into a structured, step-by-step investigation. Instead of taking a single snapshot of the truth, it builds a dynamic, growing tree of evidence. It knows when to stop searching and when to dig deeper, ensuring that the final verdict is based on a solid foundation of verified facts, not just the first thing it found on the internet.

In short: ToE doesn't just ask "Is this true?" It asks "How do we know it's true?" and builds a proof tree to answer that question.

Technical Summary: ToE – A Hierarchical and Explainable Claim Verification Framework

1. Problem Statement

The rapid proliferation of misinformation, particularly content generated or optimized by AI (e.g., via Generative Engine Optimization or GEO), poses a critical threat to information ecosystems. Large Language Models (LLMs), while powerful, rely on static training data and are prone to hallucinations. While Retrieval-Augmented Generation (RAG) and tool-calling approaches attempt to ground LLMs in real-time information, they introduce new vulnerabilities: adversarially crafted misinformation can be systematically ranked higher in retrieval results than truthful content, "poisoning" the context and leading LLMs to confidently produce false conclusions.

Existing detection methods face significant limitations:

Deep Learning-based detectors often lack generalizability across diverse topics and platforms.
LLM-based detectors (e.g., zero-shot chain-of-thought) struggle with time-sensitive news and AI-generated misinformation that is stylistically fluent but factually incorrect, especially without access to dynamic external knowledge.
Current retrieval-augmented methods often rely on static knowledge bases or lack mechanisms to actively seek counter-evidence, making them susceptible to confirmation bias and adversarial retrieval results.

2. Methodology: The Tree of Evidence (ToE) Framework

ToE is a hierarchical evidence reasoning framework designed to automate fact-checking by modeling a claim as a dynamically expanding argument tree. It integrates three core components to iteratively decompose, retrieve, and verify claims through an explainable evidence chain.

A. Core Architecture

The framework operates as an iterative loop where a claim (root node) is processed until a convergence or decisiveness threshold is met:

Dynamic Decomposition: If evidence is insufficient, an LLM decomposes the claim into sub-claims along dimensions such as who, what, when, where, why, and how.
Multi-source Retrieval: A reinforcement learning (RL) driven agent selects search actions from a discrete space (e.g., Wikipedia, arXiv, fact-checking sites, social media, counter-evidence search) based on the claim's semantic features.
Evidence Evaluation: Collected documents are parsed by an LLM to extract structured evidence snippets (stance, source credibility). An evaluation network scores the node for Veracity (probability of truth) and Reliability (confidence in the judgment).
Tree Aggregation: Scores are propagated bottom-up. If a node's reliability is high and the verdict is decisive, the branch is pruned. If reliability is low, the branch expands.

B. Theoretical Foundations (POMDP & Information Theory)

The retrieval process is modeled as a Partially Observable Markov Decision Process (POMDP):

State: Joint representation of the accumulated evidence set and claim features.
Action: Selection of a retrieval source or a stop action.
Observation: Retrieved documents processed into structured evidence.
Reward: Designed to approximate the conditional mutual information between the latent ground-truth veracity and the new observation. The reward signal is proportional to the gain in reliability ( $\Delta r_t$ ).
Theoretical Guarantee: The authors derive a formal error bound showing that if the evaluation network's reliability estimate is within $\epsilon$ of the ideal posterior, the learned policy converges to a neighborhood of the information-theoretically optimal policy. Under the assumption of submodularity in evidence accumulation, the greedy policy is guaranteed to achieve at least $(1 - 1/e)$ of the optimal non-greedy performance.

C. Key Components

RL-Driven Retrieval Agent: Uses Group Relative Policy Optimization (GRPO) to update its policy online. The agent analyzes claim categories (e.g., historical vs. scientific) and search progress to select the most informative source. It explicitly includes a "counter-evidence" action to mitigate confirmation bias.
Evidence Evaluation Agent: Employs a stance-aware multi-path attention mechanism. It encodes evidence into three parallel tracks (supporting, neutral, refuting), applies quality gating, and fuses these with claim embeddings to produce veracity and reliability scores.
Tree Management: Utilizes a rule-based fallback for interpretability and a neural aggregation network for learning. It manages expansion, pruning, and convergence based on thresholds for reliability and veracity stability.

3. Key Contributions

ToE Framework: The first algorithm to combine dynamic evidence collection with evaluation to produce an interpretable argument tree for automated fact-checking.
Theoretical Analysis: A formal derivation of error bounds for the retrieval policy, proving convergence to an optimal neighborhood under specific conditions regarding the evaluation network's accuracy.
AdvFact Dataset: The construction of an adversarial benchmark designed to simulate GEO-poisoned misinformation (using strategies like FakeGPT and PoisonedRAG) to evaluate robustness against AI-generated false claims.

4. Experimental Results

ToE was evaluated on three standard benchmarks (LIAR, PolitiFact, CheckCOVID) and the new adversarial dataset (AdvFact) using two backbone LLMs (DeepSeek-V3.2 and GPT-OSS-20B).

Performance: ToE outperformed competitive baselines (including Direct prompting, Z-CoT, STEEL, and TELLER) across all datasets. Improvements ranged from 4 to 24 percentage points.
Adversarial Robustness: The most significant gains were observed on AdvFact, where ToE achieved 64% accuracy (DeepSeek) and 54% (GPT-OSS), substantially outperforming baselines that struggled with poisoned inputs (e.g., Direct prompting dropped to 16% on AdvFact with DeepSeek).
Ablation Studies:
- Tool Space: Removing any functional group of tools (academic, counter-evidence, fact-checking, social media) degraded performance, confirming the necessity of a diverse retrieval strategy. Fact-checking databases were identified as the most critical source.
- Model Sensitivity: While baselines like Z-CoT showed high sensitivity to the underlying LLM's reasoning capacity, ToE maintained consistent improvements across both large and smaller models.
Behavioral Analysis: The retrieval agent demonstrated domain-aware behavior, prioritizing arXiv for scientific claims, fact-checking sites for political/health claims, and social media for celebrity claims.

5. Significance and Claims

The paper positions ToE as a viable path toward evidence-grounded claim verification that is resilient to GEO poisoning. Its significance lies in:

Explainability: Unlike "black box" LLM judgments, ToE provides a traceable argument tree, allowing users to review the specific evidence chain leading to a verdict.
Robustness: By actively seeking counter-evidence and dynamically expanding the search scope based on uncertainty, the framework mitigates the risks of adversarial retrieval and confirmation bias.
Generalizability: The approach is dataset-agnostic, trained on one dataset (LIAR) but demonstrating strong zero-shot transfer to other domains and adversarial settings without fine-tuning.

The authors conclude that organizing evidence into a hierarchical structure and guiding retrieval through a trained agent allows for the decomposition of complex claims into verifiable sub-arguments, offering a robust defense against the systematic surfacing of AI-generated misinformation.