Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models

1. Problem Statement

Medical errors in clinical text (e.g., diagnostic mistakes, incorrect medication) lead to significant morbidity, mortality, and financial costs. While Large Language Models (LLMs) show promise for detecting these errors, they often struggle with this specific task due to:

• Data Scarcity & Privacy: Medical data is sensitive, making it difficult to use online commercial LLMs or fine-tune models on private hospital data.
• Distribution Shift: Models trained on general data often fail on unseen, real-world clinical notes (e.g., the University of Washington dataset vs. exam-style data).
• Prompt Sensitivity: Manual prompt engineering is brittle, difficult to scale, and often fails to elicit robust reasoning in specialized domains.
• Auditability: Clinical AI systems require transparent, auditable decision-making processes, which complex fine-tuning or multi-agent debates often obscure.

The paper addresses the need for a lightweight, auditable, and privacy-preserving method to improve error detection in medical notes without expensive retraining.

2. Methodology

The authors propose a framework using Genetic-Pareto (GEPA) for automated prompt optimization.

• Dataset: The study utilizes the MEDEC benchmark (Ben Abacha et al., 2025), which includes:
  • MS (Microsoft) Subset: Synthetic clinical scenarios derived from MedQA with injected errors (used for training/validation).
  • UW (University of Washington) Subset: Real de-identified clinical notes with injected errors (used as an out-of-distribution test set to evaluate generalization).
• Models Evaluated:
  • Frontier Commercial Models: GPT-5, Grok-4, Gemini-2.5-Pro, Claude-Sonnet-4.
  • Open-Source Local Models: Qwen3 series (ranging from 0.6B to 32B parameters), capable of running on local hospital networks for privacy.
• GEPA Optimization Process:
  • Framework: Implemented via the DSPy library.
  • Mechanism: The system alternates between running an inference model on minibatches and using a reflector model to generate reflective revisions of the prompt based on task accuracy and textual feedback.
  • Pareto Selection: Prompts are retained on a Pareto frontier if they demonstrate superior performance on at least one specific validation instance, ensuring a diverse set of strategies and avoiding local minima.
  • Rich Feedback: The reflector receives natural language critiques explaining why a prediction was wrong (e.g., false positive/negative) and the correct medical reasoning, rather than just a binary signal.
  • Decoupling: The inference model (deployed for clinical notes) and the reflector model (used for optimization) can be different. This allows a powerful, costly frontier model (like GPT-5) to optimize prompts for smaller, local models (like Qwen3-4B).

3. Key Contributions

1. Systematic Evaluation: A comprehensive benchmark of both frontier commercial models and local open-source models (Qwen3) for medical error detection.
2. Automated Prompt Engineering: Demonstrating that GEPA significantly outperforms manual prompt engineering (baseline P1) across all model sizes.
3. Local Deployment Viability: Showing that small language models (SLMs) can achieve high performance when paired with optimized prompts, making them viable for secure, local hospital deployment.
4. State-of-the-Art (SOTA) Results: Achieving the highest accuracy on the MEDEC benchmark, surpassing previous SOTA methods and approaching the performance of human medical professionals.
5. Auditability: Highlighting that prompt-based solutions are inherently more auditable than fine-tuned models, as the logic is contained within explicit, reviewable text instructions.

4. Results

• Performance Gains:
  • GPT-5: Improved from a baseline accuracy of 0.669 to 0.785 (combined MS+UW).
  • Qwen3-32B: Improved from 0.578 to 0.690.
  • Generalization: Prompts optimized on the MS dataset showed even larger relative gains on the unseen UW dataset (e.g., +15.2% for GPT-5), indicating strong transferability.
• Comparison to Humans: The optimized GPT-5 (0.785) outperformed one of the two doctor baselines (0.716) and matched the other (0.796).
• Comparison to Prior Work: The GEPA-optimized GPT-5 (0.785) surpassed the previous SOTA MediFact system (0.737) and a recent multi-agent debate approach (0.788 on a small subset), while requiring only a single model call at inference time.
• Behavioral Shift: Confusion matrix analysis revealed that baseline models tended to over-predict errors (high false positives). GEPA optimization shifted models toward conservative classification, significantly improving specificity (correctly identifying error-free texts) while maintaining sensitivity.
• Reflector Dynamics: Using a stronger model (GPT-5) as a reflector generally yielded the best results for smaller models. However, for the smallest models (Qwen3-0.6B), self-reflection sometimes outperformed GPT-5 reflection, suggesting potential overfitting issues with larger reflectors on tiny models.

5. Significance and Implications

• Clinical Safety: The ability to detect errors with near-doctor-level accuracy provides a critical safeguard for both clinician-generated and AI-generated medical reports.
• Privacy & Security: The success of optimized local models (Qwen3) offers a pathway to deploy AI in "safe havens" (hospital networks) without exposing patient data to external APIs.
• Governance & Regulation: Under frameworks like ELCAP (European Society for Medical Oncology), "Background AI" systems require continuous monitoring and auditability. Prompt-based solutions are superior here because the logic is explicit, short, and easily reviewed by clinicians, unlike the "black box" nature of fine-tuned weights.
• Cost-Effectiveness: The approach avoids the computational and data costs of fine-tuning, offering a lightweight alternative to boost model performance using only inference-time optimization.

In conclusion, the paper establishes that automated prompt optimization is a critical, underutilized lever for medical AI. It transforms standard language models into highly effective, auditable, and privacy-compliant tools for clinical error detection.