💬 NLP

A Multi-Agent Approach for Claim Verification from Tabular Data Documents

The paper introduces MACE, a multi-agent framework utilizing zero-shot Chain-of-Thought reasoning across Planner, Executor, and Verifier agents to achieve state-of-the-art, interpretable claim verification from tabular data with significantly smaller models compared to existing approaches.

Original authors: Rudra Ranajee Saha, Laks V. S. Lakshmanan, Raymond T. Ng

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rudra Ranajee Saha, Laks V. S. Lakshmanan, Raymond T. Ng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery. The "mystery" is a bold claim made by a company or a scientist, like: "We recycled 55.82% of all the water we used last year!"

The "evidence" is a massive, messy spreadsheet (a table) buried inside a 50-page financial report. Your job is to check if the claim is True, False, or if there Just isn't enough info.

Doing this alone is hard. If you try to read the whole report and do the math in your head, you might get tired, miss a number, or get confused by the layout.

This paper introduces a new team of AI detectives called MACE (Multi-Agent Claim vErification). Instead of one super-smart detective trying to do everything, MACE uses a team of three specialized workers who talk to each other to solve the puzzle.

Here is how the team works, using a simple analogy:

The MACE Team: Three Specialized Agents

Think of MACE as a high-end restaurant kitchen. You don't just have one chef doing everything; you have a team with specific roles.

1. The Planner (The Head Chef / Architect)

Role: The Planner doesn't cook the food or do the math. Instead, they look at the customer's order (the Claim) and the ingredients in the fridge (the Table).
What they do: They write a recipe. They say, "Okay, to prove this claim, we need to find the 'Water Reused' row and the 'Total Water' row. Then, we need to add up the numbers for every year from 2017 to 2023. Finally, we divide the reused water by the total water."
Why it helps: They break a giant, scary problem into small, easy steps so no one gets overwhelmed.

2. The Executor (The Line Cook / Calculator)

Role: The Executor is the one who actually does the work. They take the Planner's recipe and follow it step-by-step.
What they do: They go to the table, find the specific numbers, add them up, and do the division. They write down exactly what they did: "I found 1,099,107 liters of reused water and 1,968,929 liters of total water. When I divide them, I get 55.82%."
Why it helps: They focus purely on execution. They don't worry about the big picture; they just make sure the math is right.

3. The Verifier (The Food Inspector / Quality Control)

Role: The Verifier is the strict boss who checks the work before it goes to the customer.
What they do: They read the Executor's math and ask: "Did you actually use the right numbers? Did you make a mistake? Does your answer actually match the claim?"
The Magic Loop: If the Verifier sees a mistake, they don't just say "Wrong." They send the work back to the Planner or the Executor and say, "Hey, you used the wrong row! Go fix it." This prevents errors from slipping through.

Why is this better than the old way?

The Old Way (The "Solo Detective"):
Previously, AI models tried to be a "Super Detective." They had to read the table, understand the context, do the math, and check their own work all at once.

Problem: If the model is small, it gets confused and makes mistakes. If the model is huge (like a giant brain with 235 billion "neurons"), it works well but is incredibly expensive, slow, and requires a supercomputer to run.

The MACE Way (The "Team"):
MACE uses a team of smaller, cheaper detectives.

The Result: Even with a small, cheap AI brain (like a 7-billion or 27-billion parameter model), the team approach makes them perform almost as well as the giant, expensive super-computers.
Efficiency: It's like having three small, smart interns working together is more effective than one tired genius trying to do everything alone.

The "Feedback Loop" (The Secret Sauce)

The coolest part of MACE is that the team talks to each other.

If the Executor is confused by the recipe, they ask the Planner to rewrite it.
If the Verifier sees a math error, they send it back to the Executor to fix.
This stops mistakes from piling up. In the old "one-way" systems, if the first step was wrong, the whole answer was wrong. In MACE, they catch the error before it becomes a disaster.

The Bottom Line

This paper shows that you don't need a billion-dollar supercomputer to check facts in tables. By organizing AI into a team with a clear plan, a worker, and a checker, you can get State-of-the-Art (SOTA) results (the best possible performance) while using much less memory and money.

It's a shift from "One Giant Brain" to "A Smart Team," making fact-checking faster, cheaper, and more transparent for everyone.

1. Problem Statement

The paper addresses the challenge of claim verification from tabular data documents. This task involves determining whether a natural language claim is "supported," "refuted," or if there is "not enough info" based on evidence found in structured tables (often embedded within unstructured documents like financial reports or scientific papers).

Key Challenges Identified:

Complex Reasoning: Verifying claims often requires multi-step reasoning, including identifying relevant rows/columns, understanding hierarchical structures, handling negations, and performing arithmetic operations (e.g., calculating percentages across multiple years).
Data Integration: Claims often require integrating data from both tabular sources and contextual text paragraphs.
Limitations of Current Approaches:
- Pre-training/Fine-tuning: Methods like TableGPT2 or TAT require massive datasets and computational resources, limiting generalizability to new domains.
- Single-Model Prompting: Standard Chain-of-Thought (CoT) prompting often lacks comprehensive explanations and is prone to error propagation in complex tasks.
- Unidirectional Pipelines: Decomposing tasks into linear steps (e.g., extraction $\to$ reasoning $\to$ execution) without feedback loops leads to errors compounding in later stages.

2. Methodology: The MACE Framework

The authors propose MACE (Multi-Agentic framework for Claim vErification), a prompt-based, zero-shot approach that avoids elaborate fine-tuning. It utilizes the AutoGen library to orchestrate a constrained group-chat system with three specialized agents:

Planner Agent:
- Role: Receives the claim and evidence (tables/paragraphs).
- Function: Devises a step-by-step verification strategy. It breaks down the claim into actionable steps, identifies relevant data rows/columns, handles negations, and resolves ambiguous descriptors (e.g., "significant" or "outperform") into quantitative thresholds.
- Output: An <explanation> of the strategy and a <plan> with explicit instructions for the Executor.
Executor Agent:
- Role: Implements the Planner's strategy.
- Function: Retrieves specific data cells, performs necessary computations (summation, percentage calculation), and checks trends. It distinguishes between "support," "refute," and "not enough info."
- Feedback Mechanism: If the plan is ambiguous or data is missing, the Executor can output "revise" to request clarification from the Planner, preventing premature errors.
- Output: An execution trace and a provisional verdict.
Verifier Agent:
- Role: Acts as a quality control mechanism.
- Function: Audits the Executor's explanation and provisional verdict for logical consistency and correctness. It ensures the reasoning faithfully supports the decision.
- Feedback Mechanism: If inconsistencies or errors are detected, the Verifier requests a "revise" from the Executor.
- Output: A final approved verdict or a revision request.

Workflow:
The agents operate in a sequence: User $\to$ Planner $\to$ Executor $\to$ Verifier $\to$ User. Crucially, the framework includes feedback loops (dashed arrows in Figure 1) allowing the Executor to ask the Planner for plan revisions and the Verifier to ask the Executor for corrections. This iterative refinement mitigates error propagation common in unidirectional pipelines.

3. Key Contributions

Novel Multi-Agent Architecture: Introduction of a three-agent framework (Planner, Executor, Verifier) that decomposes complex verification tasks without requiring domain-specific fine-tuning.
Zero-Shot Reasoning: Demonstrates that specialized agents using zero-shot Chain-of-Thought prompting can achieve state-of-the-art (SOTA) results, eliminating the need for expensive pre-training on specific table datasets.
Interpretability: The framework produces detailed, human-readable verification traces, explicitly showing selected rows, columns, cells, and mathematical operations, enhancing trust and accountability.
Feedback-Driven Error Correction: The inclusion of revision loops allows the system to self-correct ambiguities and inconsistencies before finalizing a verdict.
Memory Efficiency: The framework enables smaller models (27–92B parameters) to achieve performance comparable to much larger models (235B parameters) by leveraging collaborative reasoning.

4. Experimental Results

The authors evaluated MACE on four datasets across closed-domain (claims paired with specific tables) and open-domain (claims paired with document corpora requiring retrieval) settings.

Datasets:

Closed-Domain: SciTab, SEM-TAB-FACTS (SemTab).
Open-Domain: FinDVer (Financial documents), SciTab-OD (Open-domain version of SciTab).

Performance Highlights:

State-of-the-Art (SOTA) Performance: MACE achieved SOTA results on SciTab and SemTab without any fine-tuning. For example, on SemTab, MACE (using Qwen-235B) reached 0.90 Micro F1, surpassing the previous best (King001 at 0.85).
Competitive Open-Domain Results: On FinDVer and SciTab-OD, MACE matched or exceeded the performance of the best baseline models (including GPT-4o and Claude-3.5-Sonnet).
Small Model Efficiency:
- Smaller configurations (e.g., Mistral-7B or Llama-8B with MACE) achieved 80–96% of the performance of the largest baseline model (Qwen-235B).
- Specifically, Llama-8B with MACE achieved 95.6% of Qwen-235B's performance on SemTab while using only 11.9% of the memory (28B vs. 235B parameters).
Model Configuration Insights:
- For small models, using an independent, stronger Verifier (e.g., GPT-20B) significantly boosted performance.
- For large models, maintaining reasoning consistency within the same model family (same-model verifier) was more effective than using an independent verifier, which sometimes introduced domain-knowledge mismatches.

Trade-offs:

Runtime: Multi-agent coordination introduces overhead. MACE runtime was 5.3x to 27.5x longer than single-model inference depending on the model size and dataset. However, the authors argue this is justified for scenarios prioritizing accuracy and explainability over latency.

5. Significance and Impact

Democratization of High-Quality Verification: By enabling smaller, more memory-efficient models to perform at the level of massive models, MACE makes high-quality claim verification accessible on hardware that cannot support 200B+ parameter models.
Transparency: Unlike "black box" single-model approaches, MACE provides a transparent audit trail of the reasoning process, which is critical for high-stakes domains like finance and scientific research.
Generalizability: The prompt-based, zero-shot nature of the framework allows it to generalize across diverse table structures and domains without retraining, addressing a major limitation of fine-tuned models.
Future Directions: The paper suggests future work could involve specialized agents for mathematical reasoning and integrating feedback loops between retrieval and NEI (Not Enough Info) predictions.

In conclusion, MACE represents a significant shift from monolithic, fine-tuned models to collaborative, multi-agent systems for tabular reasoning, offering a compelling balance of performance, interpretability, and resource efficiency.