Multi-Agent Reasoning with Consistency Verification Improves Uncertainty Calibration in Medical MCQA

1. Problem Statement

Large Language Models (LLMs) deployed in clinical settings face a critical challenge: miscalibrated confidence scores.

• The Issue: LLMs are often systematically overconfident, assigning high probability to incorrect answers. In safety-critical medical applications, this lack of reliable uncertainty estimation prevents the system from knowing when to "defer" to a human expert.
• Existing Limitations: Traditional recalibration methods (e.g., temperature scaling, deep ensembles) often require labeled calibration data or multiple model copies, which are scarce in specialized medical domains. Furthermore, while multi-agent systems have shown accuracy improvements, their impact on confidence calibration remains unexplored.
• Goal: To develop a framework that improves both discrimination (accuracy) and calibration (reliability of confidence scores) in medical multiple-choice question answering (MCQA) without requiring ground-truth labels during inference.

2. Methodology: The MARC Framework

The authors propose MARC (Multi-Agent Reasoning with Consistency Verification), a three-component architecture designed to generate diagnoses and calibrated confidence scores using a single 7B-parameter model (Qwen2.5-7B-Instruct).

Component 1: Domain-Specific Specialist Agents

• Setup: Four independent agents are instantiated, each prompted with a specific medical specialty (Respiratory, Cardiology, Neurology, Gastroenterology).
• Process: Each agent generates an answer ($a_k$), a reasoning chain ($r_k$), and an initial confidence score ($C^{(0)}_k$) using a five-step Chain-of-Thought (CoT) protocol.
• Decoding: Greedy decoding ($T=0$) is used for the initial diagnosis to ensure deterministic outputs.

Component 2: Two-Phase Consistency Verification

Adapted from Wu et al. [2024], this component converts internal reasoning consistency into a Specialist Confidence Score (S-score).

1. Phase 1 (Extraction): The model extracts four key factual claims from the reasoning chain $r_k$.
2. Phase 2 (Verification):
   • Independent Answering: The model answers the extracted claims without the original reasoning context.
   • Reference Answering: The model answers the claims with the original reasoning context.
   • Inconsistency Measurement: The model compares the independent and reference answers using Jaccard similarity. If the similarity falls below a threshold ($\tau = 0.4$), it is flagged as inconsistent.
3. S-Score Calculation: The final S-score ($S_k$) is computed as:
   $$S_k = C^{(0)}_k \cdot (1 - I_k)$$
   Where $I_k$ is the inconsistency rate. A perfectly consistent specialist retains full confidence; a fully inconsistent one receives $S_k = 0$.

Component 3: S-Score Weighted Fusion

• Aggregation: The system aggregates votes from the four specialists. The score for a candidate answer $a$ is calculated as:
  $$\text{score}(a) = |\{k : a_k = a\}| \times \bar{S}_a$$
  Where $\bar{S}_a$ is the mean S-score of the specialists supporting answer $a$.
• Final Confidence: The final confidence ($\hat{C}$) is derived by blending the vote fraction with the mean and minimum S-scores of the supporting agents. This ensures that weak majorities receive lower confidence scores than strong consensus.

3. Experimental Setup

• Datasets: Four high-disagreement subsets were created from MedQA-USMLE (clinical vignettes, reasoning-heavy) and MedMCQA (short stems, knowledge-recall heavy). Subsets included 100 and 250 questions each.
• Ablation Study: Four configurations were tested to isolate contributions:
  1. C1: Single Specialist (Baseline).
  2. C2: Single Specialist + Two-Phase Verification (Isolates Calibration).
  3. C3: Multi-Agent Fusion (No Verification) (Isolates Accuracy).
  4. C4: Full System (Multi-Agent + Verification).
• Metrics: Accuracy, Expected Calibration Error (ECE), and Area Under the ROC Curve (AUROC).

4. Key Results

The full system (Config 4) demonstrated significant improvements across all settings:

• Calibration (ECE): The most significant finding. ECE was reduced by 49% to 74% across all four datasets compared to the single-specialist baseline.
  • Example: On MedQA-250, ECE dropped from 0.355 (baseline) to 0.091 (74.4% reduction).
  • Observation: Two-Phase Verification was identified as the primary driver of calibration improvement, even when accuracy did not increase.
• Accuracy: Multi-agent reasoning was the primary driver of accuracy gains.
  • On MedQA-250, accuracy improved from 54.4% (C1) to 59.2% (C4).
  • On MedMCQA, accuracy gains were smaller or slightly negative when adding verification to multi-agent (C3→C4), highlighting the difficulty of knowledge-recall tasks for 7B models.
• Discrimination (AUROC): The full system achieved the best AUROC on larger datasets (MedQA-250: +0.056; MedMCQA-250: +0.058).
  • Nuance: Verification alone (C2) sometimes reduced AUROC because it compressed confidence scores uniformly, improving calibration (ECE) but not necessarily the ranking of correct vs. incorrect answers. Multi-agent fusion restored this ranking ability.

5. Key Contributions

1. MARC Framework: A novel architecture combining multi-agent reasoning with label-free consistency verification to improve both accuracy and calibration.
2. Decoupling Calibration and Accuracy: The study empirically demonstrates that consistency verification is the primary mechanism for improving calibration (ECE), while multi-agent fusion is the primary mechanism for improving accuracy.
3. Robustness in Low-Knowledge Scenarios: The system maintains calibration improvements even on MedMCQA, where the 7B model lacks the specific factual knowledge to answer correctly. It successfully assigns lower confidence to wrong answers, providing a useful "defer" signal even when the model is wrong.
4. Ablation Analysis: Clear isolation of the roles of verification (calibration) vs. multi-agent reasoning (accuracy) and their interaction.

6. Significance and Limitations

• Significance:
  • Provides a practical solution for uncertainty quantification in medical AI without requiring labeled calibration data.
  • Enables safer deployment by allowing clinicians to identify cases where the AI is uncertain (low confidence) and should defer to human experts, even if the AI's absolute accuracy is imperfect.
  • Demonstrates that small LLMs (7B) can be made reliable through architectural design rather than just scaling parameters.
• Limitations:
  • Consistency $\neq$ Correctness: The system can be overconfident in internally consistent but factually wrong reasoning (a known limitation of self-verification).
  • Knowledge Gap: Performance drops on knowledge-intensive benchmarks (MedMCQA) where the model lacks specific factual recall.
  • Computational Cost: The full system requires ~16x more LLM calls than a single agent, resulting in significantly higher inference time (e.g., 833 mins vs. 116 mins for MedQA-250 on a single GPU).
  • Scope: Evaluated only on four specialties; generalizability to other domains (e.g., surgery, pharmacology) is unproven.

7. Future Directions

The authors suggest Retrieval-Augmented Generation (RAG) as the next step. By grounding the verification phase against an external medical knowledge base (e.g., PubMed), the system could penalize reasoning that contradicts established medical facts, not just reasoning that is internally inconsistent. This would address the "consistent but wrong" limitation.