Technical Summary: The Deliberative Illusion in Multi-Agent LLMs

Problem Definition

Multi-agent Large Language Model (LLM) systems are frequently designed to improve reasoning and factuality through interaction, operating under the assumption that consensus indicates successful integration of diverse evidence. However, this paper identifies a critical failure mode in deliberative tasks: the deliberative illusion. This phenomenon occurs when agents appear to reason collectively and converge on a consensus, while the underlying evidence base required for that deliberation progressively erodes.

The authors argue that for deliberative problems—particularly those involving socially contested or value-laden issues—reliability depends not merely on reaching an agreement, but on whether agents preserve the specific facts, uncertainties, and legitimate disagreements necessary to interpret the issue. Current multi-agent systems often fail this condition, producing consensus by discarding the very evidence and minority considerations that make the issue deliberative in the first place.

Methodology: DELIBTRACE

To measure this hidden process, the authors introduce DELIBTRACE, an evaluation framework that treats multi-agent deliberation as a traceable information-flow problem. The framework operates through four key stages:

1. Fact Construction and Labeling:

   • The background context of an issue is decomposed into atomic facts: short, self-contained, and verifiable propositions.
   • These facts are labeled as either critical (omission of which could change the interpretation or judgment of the issue) or non-critical.
   • This process is grounded in prior work on atomic factuality evaluation (Min et al., 2023) and framing theory (Entman et al., 1993).

2. Agent Initialization and Perspective Control:

   • Agents are initialized with situated subsets of the total atomic facts, simulating information asymmetry found in real-world deliberation (Stasser and Titus, 1985).
   • Each agent is assigned a prior stance (YES/NO) to capture initial disagreement.
   • This setup ensures that the system collectively holds the full evidence base, but individual agents only possess partial views.

3. Controlled Deliberation:

   • Agents exchange responses under three distinct communication topologies: fully connected, binary tree, and chain.
   • The framework tracks the survival of facts and the evolution of stances across multiple discussion rounds.

4. Evaluation Metrics:

   • Factual Survival: Measured using the Jaccard index at both the system level (union of facts retained by any agent) and the agent level (facts retained by individual agents). This distinguishes between overall retention and the retention of issue-critical facts.
   • Stance Collapse: Measured via stance entropy. High entropy indicates diverse positions, while low entropy indicates convergence. The framework specifically looks for "over-convergence" where entropy drops significantly while factual retention also declines.

Key Results

The study was conducted across two domains (ETHICS, derived from Reddit "Am I the Asshole?" scenarios, and NEWS, derived from public-issue articles) using three representative LLM families: GPT-4.1, Gemini-3, and Qwen-3.5.

• Substantial Factual Attrition: Multi-agent discussion leads to a progressive loss of issue-critical facts. By the final round, systems erased up to 72% of issue-critical atomic facts at the system level and up to 84.8% at the agent level. This loss is amplified by inter-agent communication; agents lose significantly more evidence when responding to and summarizing each other compared to rounds where they revise in isolation.
• Stance Homogenization: As discussion proceeds, stance entropy decreases sharply, indicating that agents converge toward shared positions. This convergence often occurs while the factual basis for those positions shrinks.
• The Deliberative Illusion: The paper demonstrates that agents can agree more while knowing less. The resulting consensus is often a "biased conclusion" that omits concrete evidence, conditions, and distinctions necessary for faithful interpretation.
• Downstream Consequences:
  • Misleading Reconstruction: Retained facts often become too abstract to faithfully reconstruct the original issue. In the NEWS domain, the intersection of facts shared by all agents led to misleading reconstructions in approximately 90% of cases by Round 3.
  • Judgment Errors: Factual attrition caused the system's final judgment to be incorrect in 19.2% of cases where the base model would have been correct given the full context.
  • Prior Anchoring: Final consensus often aligns more closely with the base model's prior tendencies (match rates of 0.65–0.76) than with the integrated evidence, suggesting discussion pulls groups toward pre-existing model biases rather than synthesizing new insights.
  • Vulnerability to Misinformation: A single malicious agent injecting misinformation into the shrinking shared context can cause that misinformation to persist in 58.9% of final system outputs, even when truthful agents collectively possess the evidence to refute it.

Significance and Claims

The paper claims that consensus is an unreliable proxy for deliberative quality. The primary contribution is the identification and measurement of the "deliberative illusion," where the appearance of collective reasoning masks the erosion of the factual foundation.

The authors assert that current multi-agent evaluations, which often focus on final-answer accuracy or consistency, fail to capture whether the process of deliberation preserves the diversity and evidence required for trustworthy outcomes. They argue that for multi-agent systems to be reliable in deliberative settings, future evaluations must measure:

1. Which specific facts survive interaction.
2. Whether uncertainties and legitimate disagreements are preserved.
3. Whether the final agreement is based on integrated evidence or merely the result of factual attrition and stance homogenization.

The paper concludes that while multi-agent interaction holds promise, current implementations often degrade the epistemic conditions necessary for that promise to be realized, necessitating new protocols that prioritize the preservation of evidence over the speed of convergence.