💬 NLP

MedGuards: Multi-Agent System for Reliable Medical Error Detection and Correction

The paper introduces MedGuards, a multi-agent framework that enhances the safety of Large Language Models in healthcare by detecting, localizing, and correcting medical errors through specialized agents and a confidence-guided arbitration mechanism, while also proposing a new Keyword-Prioritized Correction Score (KPCS) metric to better evaluate critical keyword accuracy.

Original authors: Congbo Ma, Hu Wang, Yichun Zhang, Farah E. Shamout

Published 2026-06-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Congbo Ma, Hu Wang, Yichun Zhang, Farah E. Shamout

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor writing a patient's medical record. You use a super-smart AI assistant to help you draft the notes. This AI is like a brilliant but occasionally distracted medical student; it writes beautifully and fluently, but sometimes it accidentally swaps a critical detail—like writing "Hepatitis A" instead of "Schistosoma mansoni." In the real world, that tiny mistake could lead to the wrong treatment, which is dangerous.

The paper introduces MedGuards, a new system designed to act as a "safety net" for these AI-generated medical notes. Instead of trusting a single AI to check its own work, MedGuards uses a team of specialized AI agents working together, much like a hospital's "Grand Rounds" or a panel of expert consultants.

Here is how MedGuards works, broken down into simple concepts:

1. The Team of Specialists (Multi-Agent System)

Instead of one AI trying to do everything at once, MedGuards splits the job into three distinct roles, similar to a construction crew:

The Detector (The Spotter): This agent's only job is to look at the text and shout, "Hey, I think there's a mistake here!" or "Everything looks fine."
The Localizer (The Inspector): If a mistake is found, this agent points exactly to the specific sentence where the error lives.
The Corrector (The Fixer): This agent rewrites that specific sentence to make it medically accurate.

2. The "Second Opinion" Rule (Self-Consistency)

To make sure the team doesn't make a collective mistake, MedGuards uses a "two-eyes-are-better-than-one" approach.

Two different agents independently check the text.
If they agree: Great! They move on.
If they disagree: This is where it gets clever. A third agent, the Arbiter, steps in. Think of this Arbiter as a senior judge. It doesn't just guess; it looks at the reasoning and the confidence scores (how sure each agent feels) of the first two agents. It listens to their arguments and makes the final call. This ensures that even if the AI is unsure, the system doesn't just guess; it deliberates.

3. The "Keyword" Scorecard (KPCS)

The paper argues that standard ways of grading AI writing (like checking how many words match) are flawed for medicine.

The Problem: Imagine the AI writes, "The patient has a broken leg," but the correct note should say, "The patient has a broken heart." Standard grading might give this a high score because the sentence structure is perfect, even though the meaning is deadly wrong.
The Solution: MedGuards introduces a new score called KPCS (Keyword-Prioritized Correction Score). It acts like a safety inspector who ignores the pretty words and only cares if the critical medical terms (like specific diseases, drugs, or body parts) are correct. If the critical keywords are wrong, the score drops, no matter how well the rest of the sentence flows.

4. No New Training Needed

A major benefit of MedGuards is that it doesn't require retraining the AI models. It works like a "plug-and-play" add-on. You can take an existing medical AI and slot MedGuards in front of it to make it safer and more reliable immediately.

What the Results Show

The authors tested this system on medical notes in three languages (English, Arabic, and Chinese). They found that:

MedGuards consistently found and fixed more errors than previous methods.
It worked well even when paired with different types of AI models (from smaller to very large ones).
The "Judge" (Arbiter) was used more often when the task was hard (finding exactly which sentence was wrong), proving that the team-up approach helps solve tricky problems.

In summary: MedGuards is a framework that turns a single, potentially error-prone AI into a collaborative team of experts with a built-in dispute resolution system, ensuring that medical notes are not just grammatically correct, but clinically safe.

Technical Summary: MedGuards – A Multi-Agent System for Reliable Medical Error Detection and Correction

Problem Statement

As Large Language Models (LLMs) are increasingly deployed in healthcare settings for tasks such as clinical documentation, diagnostic assistance, and treatment recommendations, the risk of generating or retaining medical errors poses significant threats to patient safety. Existing error detection and correction methods, including automated checks and heuristic-based approaches, often fail to generalize across unseen datasets. Furthermore, current quality control mechanisms primarily assess outputs at the semantic level, which is insufficient for addressing errors requiring deep structural understanding and clinical reasoning. A critical gap exists in evaluation metrics as well: standard NLP metrics (e.g., BLEU, ROUGE) treat all tokens equally, failing to account for the varying importance of specific clinical entities (e.g., causal organisms, diagnoses). Consequently, a model might achieve high similarity scores while generating fundamentally incorrect clinical facts.

Methodology

The authors propose MedGuards, a framework that treats medical error detection and correction as a multi-agent in-context learning (ICL) task. Rather than relying on a single model or requiring additional training of base LLMs, MedGuards orchestrates specialized agents to perform step-wise reasoning. The system is grounded in three core principles:

Chain-of-Thought (CoT) Decomposition: The overall task is decomposed into three distinct reasoning stages:
- Error Detection ( $f_{Det}$ ): Determining if an error exists in the input clinical note.
- Error Localization ( $f_{Loc}$ ): Identifying the specific sentence(s) containing the error.
- Error Correction ( $f_{Corr}$ ): Generating a corrected version of the erroneous text.
Multi-Agent Self-Consistency: To enhance reliability under uncertainty, the system employs multiple agents (typically two base agents) that independently reason over the same input. Their outputs are compared to reach a consensus. If the base agents disagree, a third arbitration agent is activated. This agent adjudicates the conflict by analyzing the original inputs, candidate sets, reasoning traces, and confidence scores provided by the base agents. This design extends the concept of self-consistency from single-model sampling to a multi-agent setting, treating disagreement as a signal for higher-order deliberation rather than noise.
ICL-Based Arbitration: The arbitration process is formulated as an ICL problem. The arbitration agent does not undergo parameter updates; instead, it conditions its decision on the context provided in the prompt, which includes the reasoning traces and confidence scores of the conflicting agents. This allows the system to synthesize final decisions from conflicting rationales dynamically.

Workflow:

Detection: Two agents ( $Agent_{Det}^{1,2}$ ) output binary predictions ( $d \in \{0,1\}$ ) with reasoning and confidence. If they agree, the result is final. If they disagree, $Agent_{Det}^{3}$ arbitrates.
Localization: Two agents ( $Agent_{Loc}^{1,2}$ ) identify the erroneous sentence. If they disagree, $Agent_{Loc}^{3}$ arbitrates. The system then aligns the predicted erroneous sentence with the original text using character-level string similarity (SequenceMatcher) to ensure precise localization, avoiding semantic hallucinations.
Correction: A dedicated correction agent generates the final corrected text based on the localized error and the original context.

Key Contributions

The paper introduces four primary contributions:

MedGuards Framework: The first multi-agent system tailored specifically for medical error detection and correction. It features a plug-and-play design with three collaborative modules (detection, localization, correction) that can be integrated with existing LLM-based medical agents without retraining.
Confidence-Guided ICL Arbitration: A mechanism to resolve disagreements between agents by leveraging reasoning traces and confidence scores, improving robustness in uncertain scenarios.
Keyword-Prioritized Correction Score (KPCS): A novel evaluation metric designed to address the limitations of token-equality assumptions in standard metrics. KPCS explicitly checks whether critical clinical keywords (e.g., diagnoses, medications) from the reference text are correctly generated in the output. It combines a keyword presence check with standard similarity metrics (ROUGE-1, BERTScore, BLEURT) using a weighted formula:
$KPCS(S_c, S_r) = \alpha \cdot K(S_c, S_r) + (1 - \alpha) \cdot M(S_c, S_r)$
where $K$ is the keyword ratio and $M$ is the average similarity score.
Comprehensive Evaluation: Extensive experiments across four datasets (including MEDEC and MedErrBench) covering three languages (English, Arabic, Chinese) and multiple base LLMs.

Experimental Results

The authors evaluated MedGuards against state-of-the-art baselines (e.g., knowlab_AIMed, EM-mixer, Medifact) and various LLM backbones (e.g., Gemini 2.0/2.5, GPT-4o-mini, Doubao-1.5, Deepseek-V3, Llama-3.3).

Performance Gains: MedGuards consistently outperformed existing methods and plain LLM backbones across all sub-tasks (detection, localization, correction). For instance, on the MEDEC dataset, MedGuards (using Doubao-1.5 as the base) achieved 0.770 accuracy in error detection and 0.716 in localization, significantly surpassing the best baseline (Medifact at 0.737 detection).
Metric Improvements: The framework showed substantial improvements in correction quality metrics (ROUGE-1, BERTScore, BLEURT). Notably, KPCS scores improved dramatically (e.g., from 0.210 to 0.461 for GPT-4o-mini), indicating that the corrections preserved critical clinical key-phrases better than baselines.
Generalizability: The system demonstrated robustness across different languages and base models, narrowing the performance gap between weaker and stronger LLMs.
Ablation Studies: Results confirmed that including reasoning traces and confidence scores in the arbitration process consistently improved performance. Furthermore, the "soft matching" strategy for localization (aligning based on string similarity) proved more robust than "hard matching" (exact index prediction).
Self-Consistency Dynamics: Analysis revealed that while detection often relies on consensus, localization frequently requires arbitration (approx. 30-38% of cases), highlighting the inherent ambiguity in pinpointing specific erroneous sentences.

Significance and Claims

The paper claims that MedGuards offers a systematic solution for the safer deployment of LLMs in real-world healthcare applications. By decomposing the complex task of error correction into interpretable, multi-agent steps and introducing a confidence-guided arbitration mechanism, the framework enhances reliability without the need for additional model training. The introduction of KPCS addresses a critical gap in evaluation, ensuring that corrections are not just fluent but also clinically accurate regarding domain-critical entities. The authors position MedGuards as a step toward trustworthy integration of generative models into clinical environments, providing a scalable and adaptable approach to maintaining patient safety in AI-assisted healthcare.