MAC: Masked Agent Collaboration Boosts Large Language Model Medical Decision-Making

Technical Summary: Masked Agent Collaboration (MAC) for Medical Decision-Making

Problem Statement
While Large Language Models (LLMs) and Multi-Agent Systems (MAS) show promise in healthcare, current MAS-based approaches face two critical limitations in medical decision-making scenarios:

1. Lack of Systematic Agent Construction: Existing methods often select collaborative agents based solely on standalone performance metrics (e.g., accuracy) or diversity, ignoring essential operational trade-offs such as model size, inference time, and throughput. This leads to suboptimal ensembles that are either computationally prohibitive for clinical deployment or lack the necessary diagnostic reliability.
2. Rigid Collaboration Patterns: Current MAS frameworks typically employ static collaboration patterns where all agents contribute to every layer of aggregation. This rigidity makes them vulnerable to "collaboration failures," where overconfident but incorrect outputs from specific agents propagate misinformation, degrading overall performance. In medical contexts, where data scarcity can lead to hallucinations, this propagation of inconsistent outputs undermines the system's utility.

Methodology: The MAC Framework
The authors propose the Masked Agent Collaboration (MAC) framework, which introduces a dynamic, adaptive pipeline to enhance medical decision-making. The framework consists of two primary mechanisms:

1. Pareto-Optimal Agent Construction:

   • Instead of arbitrary selection, MAC analyzes a pool of candidate LLMs using a multi-objective optimization approach based on four key factors: Model Size, Inference Time, Diversity Score, and Throughput Ratio.
   • The Diversity Score is calculated by measuring the semantic similarity between pairwise outputs generated by a single LLM on the same query (using Levenshtein distance on sliding windows).
   • The system identifies the Pareto frontier to select a cohort of agents that balances efficiency (size, time, throughput) and capability (diversity). This ensures the selected ensemble is both high-performing and practically deployable in resource-constrained clinical environments.

2. Adaptive Progressive Propagation via Cross-Consistency Maximization:

   • Unlike static aggregation, MAC employs a layer-wise iterative process where agents generate outputs based on the aggregated inputs of unmasked agents from the previous layer.
   • Cross-Consistency (CC) Calculation: The system computes pairwise similarity between the outputs of all active agents.
   • Dynamic Masking: In each layer, the agent with the lowest minimum pairwise similarity (lowest cross-consistency) is identified and masked (removed) from the active set. Its output is excluded from the next layer's aggregation.
   • This process repeats for a predefined number of layers, progressively pruning inconsistent or anomalous viewpoints. The remaining agents refine their reasoning based on the consensus of the "unmasked" cohort, simulating a self-adjusting discussion that filters out noise and hallucinations.

Key Contributions

• Identification of Failure Modes: The paper highlights that current MAS frameworks often fail in medical scenarios because they lack mechanisms to handle agent overconfidence and misinformation propagation, leading to performance degradation compared to individual LLMs.
• Pareto-Optimal Construction Mechanism: A novel pipeline for selecting collaborative agents that rigorously trades off performance against computational constraints (size, time, throughput), ensuring real-world applicability.
• Cross-Consistency Maximization (CCM): An iterative masking strategy that dynamically removes the least consistent agent at each layer. This mitigates performance degradation by filtering inconsistent outputs during adaptive progressive propagation.
• Empirical Validation: Extensive experiments on three specialized medical datasets (NEJMQA, MMLUPH, MedQA) and a general reasoning dataset (MMLU-Pro Math).

Results

• Performance Gains: On the NEJMQA dataset, the MAC framework (composed of open-access 14B–32B models) outperformed a multi-agent collaboration model composed of 70B–141B open-access LLMs by 16.55% and surpassed the closed-source Claude Haiku 4.5 by 5.29%.
• Efficiency: Compared to the MoA baseline, MAC achieved a 6.11% higher accuracy while reducing memory usage by 70,206 MB and running time by 41,855.52 seconds, demonstrating superior scalability.
• Robustness: Ablation studies confirmed that both the Pareto-optimal construction and the cross-consistency masking strategies are essential, with their combination yielding the highest improvements across Accuracy, F1, Precision, Specificity, and MCC metrics.
• Generalizability: The framework also demonstrated strong performance on open-ended medical scenarios and mathematical reasoning tasks, suggesting the underlying logic of iterative pruning applies beyond strict multiple-choice medical questions.

Significance and Claims
The paper claims that MAC provides a highly efficient, deployable, and clinically robust solution for real-world medical decision-making. By moving away from static, resource-heavy ensembles toward a dynamic, consistency-driven pruning mechanism, MAC effectively bridges the gap between AI development and clinical deployment. The authors emphasize that their approach complements rather than replaces physicians, offering a tool that can reduce errors and optimize diagnosis pathways, particularly in settings where experienced physicians are scarce. The work underscores that carefully curated, smaller open-access models, when guided by a rigorous collaboration protocol, can outperform massive proprietary models in specialized medical domains.