Technical Summary: The Consistency Illusion in Multi-Agent Medical QA

1. Problem Statement

Multi-agent Large Language Model (LLM) systems are increasingly deployed for medical question answering (QA), operating under the assumption that answer-level consensus (multiple agents agreeing on a final diagnosis or treatment) serves as a reliable proxy for reasoning-level alignment. Frameworks such as MedAgents and MDAgents report accuracy gains over single-agent baselines by leveraging debate and voting mechanisms.

However, this paper argues that this trust assumption is incomplete. The authors identify a failure mode termed the Consistency Illusion: a phenomenon where multi-agent debate reduces detectable contradictions between agents while simultaneously decreasing the semantic similarity of their underlying reasoning chains. In this state, agents appear to agree more frequently but reason less consistently. As illustrated in the paper's motivating example, three agents may independently converge on the correct clinical answer (e.g., atropine for symptomatic bradycardia) while invoking mutually exclusive pharmacological mechanisms (e.g., $\beta_1$-agonism vs. M2-blockade vs. acetylcholinesterase inhibition).

Existing evaluation pipelines primarily measure macroscopic answer accuracy or single-agent reasoning faithfulness, leaving cross-agent reasoning alignment unmeasured. Consequently, systems may produce reliable-looking consensus based on medically incompatible rationales, a critical risk in safety-critical domains.

2. Methodology

2.1 CARA: Cross-Agent Reasoning Alignment

To quantify this phenomenon, the authors introduce CARA, a family of automated metrics designed to measure whether agents who agree on an answer also agree on the reasoning behind it. CARA operates post-hoc on system traces without requiring additional LLM calls.

• Agreement Set Definition: For a given question, CARA isolates the subset of agents $S(q)$ that voted for the majority answer.
• Step-Level Decomposition: Each agent's rationale is decomposed into reasoning steps using a deterministic text-segmentation pipeline.
• Hybrid Alignment Metric (CARA-HYB): The metric computes pairwise alignment between steps of different agents using a hybrid of:
  • NLI-based Contradiction Detection: Using a DeBERTa-v3 model to flag contradictions ($P_{NLI} > \tau$).
  • Embedding Similarity: Using Stella embeddings to calculate cosine similarity for non-contradictory steps.
  • Logic: Contradictions override similarity (a step pair is scored as -1 if contradictory, otherwise by cosine similarity).
• Aggregation: Scores are symmetrized across agent pairs and aggregated at the question and corpus levels.
• Diagnostics: The framework also reports the Contradiction Rate (CR), the fraction of best-matched step pairs flagged as contradictions.

2.2 Grounded Debate Protocol (GDP)

To test whether the consistency illusion is a fundamental flaw of multi-agent interaction or a protocol deficiency, the authors propose GDP, a prompt-level intervention. GDP enforces a structured communication format requiring agents to output three machine-parseable fields for each reasoning step:

1. CLAIM: A singular, falsifiable clinical assertion.
2. GROUND: A named medical fact, mechanism, or guideline supporting the claim (e.g., specific drug targets, receptor pathways).
3. STANCE: (In debate rounds) An explicit position (AGREE, DISAGREE, EXTEND) toward another agent's claim, requiring a counter-GROUND for disagreements.

GDP includes an "anti-sycophancy" rule, instructing agents to change answers only if they identify a factual error or receive a more compelling ground, rather than simply adopting the majority view.

2.3 Experimental Setup

• Datasets: MedQA-USMLE (D1, N=500, closed-form MCQs) and MedThink-Bench (D2, N=499, open-ended reasoning with expert annotations).
• Backbones: Qwen 2.5 72B Instruct (primary) and Llama 3.3 70B Instruct (replication).
• Systems:
  • M4: Independent voting control (no debate).
  • M3: Standard symmetric debate (Du et al., 2024).
  • M3-GDP: Standard debate architecture with GDP prompt constraints.
• Validation: CARA scores were validated against GPT-4o as a judge and three medically-trained human annotators, showing strong monotonicity and correlation.

3. Key Results

3.1 The Consistency Illusion

Applying CARA to standard debate (M3) revealed the illusion:

• Contradiction Rate (CR) Decreased: Debate reduced explicit contradictions between agents (e.g., D1 CR dropped from 0.104 to 0.035).
• Semantic Similarity (SIM) Decreased: Simultaneously, the semantic similarity of reasoning chains dropped (e.g., D1 SIM dropped from 0.801 to 0.787).
• Interpretation: Standard debate suppresses surface-level disagreement (smoothing contradictions) but causes reasoning to diverge further, creating a false sense of consensus. On D2, this effect was significant ($d = -0.30$ for Qwen, $d = -1.32$ for Llama).

3.2 Effectiveness of Grounded Debate (GDP)

GDP reversed this trend, producing the "true alignment" signature:

• CR Decreased: Contradictions were reduced (e.g., D1 CR: 0.117 $\to$ 0.098).
• SIM Increased: Semantic similarity of reasoning chains significantly improved (e.g., D1 SIM: 0.835 $\to$ 0.912).
• Magnitude: GDP produced large, consistent alignment gains across all conditions (Tier A effect sizes: $d = +1.43$ to $+1.99$).
• Expert Alignment: CARA-GT (alignment with expert Scoring Points) showed GDP moved agents closer to expert reasoning ($d = +0.32$), whereas standard debate moved them away ($d = -0.21$).

3.3 Failure Mode Analysis

Analysis of the lowest-CARA cases revealed that GDP structurally eliminated two severe failure modes:

1. Complementary Reasoning (FM1): Agents using different, non-contradictory paths to the same answer (eliminated: 23 $\to$ 0 on D1).
2. Sycophantic Convergence (FM4): Agents adopting the majority answer with zero reasoning steps (eliminated: 11 $\to$ 0 on D1).
   Residual misalignment under GDP shifted to "Partial Overlap" (FM6), a low-severity baseline.

3.4 Accuracy Considerations

The paper explicitly notes that CARA measures alignment, not correctness.

• Standard Debate: Did not significantly change answer accuracy compared to independent voting.
• GDP: Did not significantly improve accuracy; in some cases, it showed a slight, non-significant decrease.
• Conclusion: GDP improves the reliability of the reasoning process without necessarily altering the accuracy of the final answer in these benchmarks.

4. Significance and Claims

The paper claims to identify and quantify a previously unmeasured failure mode in multi-agent medical QA: the Consistency Illusion. The authors argue that the prevailing assumption—that multi-agent consensus implies reliable clinical reasoning—is flawed because surface agreement can mask divergent, medically incompatible rationales.

The significance of the work lies in:

1. Metric Innovation: Introducing CARA as a necessary audit tool for safety-critical domains, measuring reasoning alignment alongside accuracy.
2. Protocol Intervention: Demonstrating that lightweight, prompt-level grounding (GDP) can substantially improve cross-agent reasoning alignment without modifying system architecture or increasing LLM call budgets.
3. Safety Implications: Highlighting that in high-stakes settings, "agreement" is insufficient; systems must be audited for consistent reasoning to ensure that consensus is not merely a statistical artifact of sycophancy or vague smoothing.

The authors maintain a modest stance, clarifying that GDP is an alignment-auditing intervention, not a guaranteed accuracy booster, and that the findings are specific to the tested debate topologies and open-weight models. They position GDP as a diagnostic and auditing intervention to close the gap between surface consensus and reasoning reliability.