💬 NLP

Multi-Agent Causal Discovery Using Large Language Models

The paper introduces MAC, a multi-agent framework that enhances causal discovery by combining autonomous statistical algorithm selection with an adversarial LLM debate over metadata, achieving superior performance and robustness across diverse benchmarks compared to existing statistical and single-agent LLM methods.

Original authors: Hao Duong Le, Xin Xia, Haijie Xu, Chen Zhang

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hao Duong Le, Xin Xia, Haijie Xu, Chen Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Finding the "Why"

Imagine you are a detective trying to figure out why a car broke down. You have two main tools:

The Data Log: A spreadsheet of numbers showing the car's speed, temperature, and fuel levels over time.
The Mechanic's Manual: A book of rules, common sense, and expert knowledge about how cars usually work.

Traditional methods only look at the Data Log. They are great at finding patterns in numbers but often miss the "big picture" or get confused if there isn't enough data.
On the other hand, recent AI methods (using Large Language Models or LLMs) act like a single expert mechanic reading the manual. They are smart and know a lot, but they can be tricky. Sometimes, they just "guess" based on what they've memorized from the internet, rather than actually looking at the specific car in front of them. They might say, "It's definitely the battery," because that's a common answer, even if the data says otherwise.

The Solution: The MAC Framework

The authors created a new system called MAC (Multi-Agent Causal Discovery). Instead of relying on one detective or one mechanic, MAC sets up a team debate that combines the best of both worlds: hard data and expert knowledge.

Think of MAC as a two-stage courtroom trial.

Stage 1: The "Data Detective" (Debate-Coding Module)

First, the system needs to look at the raw numbers (the Data Log). But which math tool should it use? There are many different statistical formulas, and no single one works best for every car.

The Debate: Before doing the math, three AI agents (a "Yes" lawyer, a "No" lawyer, and a "Judge") have a quick debate. They look at a small sample of the data and the manual.
The Verdict: They argue over which statistical formula is best suited for this specific dataset.
The Action: Once they agree on the best formula, they run it on the full data. This creates a rough draft of the causal graph (the map of what causes what). This step ensures the answer is grounded in actual numbers, not just guesses.

Stage 2: The "Expert Panel" (Meta-Debate Module)

Now, the system has a rough draft from the numbers, but it might still be missing some context (like "this car was driven in the rain"). This is where the Metadata (the expert manual) comes in.

The Debate: The rough draft is put on trial.
- The Affirmative Agent says: "This graph is correct because the data supports it."
- The Negative Agent says: "Wait! According to the expert manual, this relationship doesn't make sense. Here is a better explanation."
- The Judge listens to both sides, checks the evidence against the rules in the manual, and picks the winner.
The Result: The graph is refined. The AI doesn't just trust one opinion; it forces the "Yes" and "No" sides to argue until the most logical, evidence-backed conclusion remains.

The Glue: "Meta Fusion"

How do you talk to a computer that speaks numbers and a computer that speaks words? The paper introduces a "Meta Fusion" mechanism. It takes the rough graph from Stage 1 and translates it into plain English sentences (e.g., "Speed causes Heat"). This allows the Stage 2 debate to happen naturally, mixing the hard numbers with the expert rules.

Why This Matters (The Results)

The authors tested this system on five different "cases" (datasets ranging from car fuel efficiency to earthquake detection). They compared MAC against:

Old-school math-only detectives.
Single-AI experts.
Other AI methods.

The Outcome:
MAC won the most often. It was the only system that could perfectly reconstruct the "Earthquake" graph (a classic test case) and consistently ranked #1 across different types of AI models.

The Trade-off: Cost vs. Quality

The paper notes that running a full team debate costs more "tokens" (computational money) than asking a single AI a simple question.

For simple puzzles: You can use a "Single-Agent" version of MAC (just one lawyer and a judge) to save money.
For complex puzzles: The full three-person debate is necessary to get the right answer, especially when the data is messy or the rules are unclear.

Summary

In short, MAC stops AI from just "guessing" based on memory. It forces the AI to:

Check the math first (to ensure it's grounded in reality).
Argue the logic second (to ensure it makes sense with expert knowledge).
Let a Judge decide (to pick the best answer).

It's like moving from asking one person for advice to holding a town hall meeting where the data speaks, the experts speak, and a neutral referee ensures the final decision is the truth.

Technical Summary: Multi-Agent Causal Discovery Using Large Language Models (MAC)

Problem Statement

Causal discovery aims to identify causal relationships between variables, a fundamental challenge across scientific disciplines. Traditional Statistical Causal Discovery (SCD) methods rely exclusively on observational data, often failing in large-scale systems where sufficient structured data is infeasible to obtain. Furthermore, these methods ignore valuable contextual metadata (e.g., variable semantics, domain knowledge). Conversely, recent Large Language Model (LLM)-based approaches leverage metadata but typically treat the LLM as a single agent. This single-agent paradigm is vulnerable to "causal parrots"—models that rely on memorized or biased associations rather than genuine reasoning, leading to unreliable judgments when faced with complex or dense causal graphs.

Methodology: The MAC Framework

The authors propose MAC (Multi-Agent Causal Discovery Framework), which reframes causal graph discovery as a multi-agent debate coupled with the autonomous selection of statistical algorithms. MAC integrates two complementary modules bridged by a Meta Fusion mechanism:

1. Debate-Coding Module (DCM)

The DCM addresses the issue that purely text-based reasoning lacks grounding in observed data, while no single statistical algorithm is optimal for all datasets.

Algorithm Selection: The DCM embeds a Meta-Debate Module (MDM) to debate and select the most suitable SCD algorithm (e.g., PC, GES, FCI) based on a small subset of structured data ( $S$ ) and metadata ( $I$ ).
Execution: The selected algorithm is executed on the full structured data ( $O$ ) to generate an initial causal graph ( $G_C$ ). This ensures the initial graph is grounded in empirical evidence.
Meta Fusion: The resulting graph $G_C$ is converted into compact, human-readable causal constraints ( $I_C$ ) (e.g., directed edges and implied orderings). These constraints serve as "causal metadata" extracted from the data.

2. Meta-Debate Module (MDM)

The MDM addresses the unreliability of single-LLM judgments by introducing an adversarial debate structure.

Agents: The module employs three specialized agents:
- Causal Affirmative Debater: Proposes a candidate causal graph (or algorithm).
- Causal Negative Debater: Proposes a plausible alternative, specifically targeting disputed decisions to surface conflicting hypotheses.
- Causal Judge: Adjudicates between the proposals based on explicit, evidence-derived criteria (conditional independence, collider structures, and metadata alignment).
Workflow:
- Stage 1 (DCM): The MDM selects the SCD algorithm.
- Stage 2 (Refinement): The MDM takes the initial graph $G_C$ (via its converted constraints $I_C$ ) and the original metadata $I$ as input. The agents debate to refine the graph, adjudicating competing hypotheses against the combined evidence.

3. Integration

The framework operates in an end-to-end pipeline:

DCM selects an algorithm and generates an initial data-grounded graph.
Meta Fusion translates this graph into textual constraints.
MDM uses these constraints alongside original metadata to refine the graph through adversarial debate, producing the final Directed Acyclic Graph (DAG).

Key Contributions

Framework Design: The paper introduces MAC, a novel framework that couples adversarial multi-agent LLM debate with the autonomous selection of statistical causal discovery algorithms. This integrates data-grounded statistical precision with metadata-driven, expert-level reasoning.
Modular Architecture: The authors propose two specific modules:
- Meta-Debate Module (MDM): Counters single-LLM bias by having agents adversarially propose, challenge, and adjudicate causal graphs against metadata.
- Debate-Coding Module (DCM): Grounds the graph in data by autonomously selecting and executing the best-suited SCD algorithm.
Performance: The framework demonstrates superior aggregate performance across five benchmark datasets and three evaluation metrics (F1, SHD, NHD) compared to both traditional statistical baselines and existing LLM-based methods.

Experimental Results

The authors evaluated MAC on five datasets (Child, Auto, Earthquake, Cancer, Survey) using three backbone LLMs (Gemini-2.0-Flash, DeepSeek-R1, GPT-4o) and compared it against five statistical methods and four LLM-based baselines.

Overall Performance: MAC achieved the best aggregate performance across 15 evaluation points (5 datasets × 3 metrics).
Specific Achievements:
- With Gemini-2.0-Flash, MAC ranked first on 10 of 15 evaluation points, including a perfect reconstruction (F1=1.00, SHD=0, NHD=0) of the Earthquake graph.
- With DeepSeek-R1, MAC ranked first on 9 points; with GPT-4o, it ranked first on 7 points.
Ablation Studies:
- Module Comparison: Combining DCM and MDM generally outperformed using either module in isolation, though MDM alone performed best on the Earthquake dataset, suggesting metadata can sometimes suffice for simple graphs.
- Debate Rounds: Performance typically converged after a single round of debate; additional rounds introduced fluctuation rather than consistent improvement.
- Cost vs. Performance: A "Single-Agent MAC" variant (removing the Negative agent and Judge) offered a cost-effective alternative for simple graphs (5 nodes), recovering much of the gain at roughly half the cost. However, the full multi-agent debate was essential for larger or more ambiguous graphs (e.g., Survey, Auto), where it significantly improved F1 scores compared to the single-agent variant.

Significance and Claims

The paper claims that MAC provides a generalizable template for machine learning tasks involving metadata, extending beyond causal discovery to areas like classification and anomaly detection. The authors emphasize that their approach mitigates the "causal parrot" problem by grounding LLM reasoning in statistical evidence (via DCM) and rigorously testing hypotheses through adversarial debate (via MDM).

The authors remain modest regarding limitations, acknowledging that:

The current framework models only DAGs and does not handle confounders or cycles.
It relies solely on observational data without interventional validation.
The multi-agent debate incurs substantial computational overhead.
Evaluation is currently limited to established benchmarks with relatively small graphs (5–20 nodes), and the framework's performance on novel, large-scale, data-scarce regimes requires further validation to ensure reasoning is not merely recall of memorized canonical networks.