Technical Summary: LLM-Guided Evolution for Medical Decision Pipelines

Problem Statement

Adapting Large Language Models (LLMs) to specialized clinical workflows typically relies on two expensive approaches: computationally intensive fine-tuning requiring large annotated datasets and GPU resources, or manual prompt and pipeline engineering that depends on iterative human-in-the-loop refinement. The latter is often time-consuming, difficult to reproduce, and inconsistent across practitioners. While LLM-guided MAP-Elites optimization has emerged as an inference-time alternative for general domains, its application to safety-sensitive medical decision pipelines remains limited. Existing studies primarily target general benchmarks or isolated prompts, rather than optimizing executable decision procedures where prompts, control logic, interaction policies, and output constraints can all evolve simultaneously.

Methodology

The authors propose using LLM-guided MAP-Elites evolution as an inference-time framework to discover medical decision strategies without fine-tuning the underlying task models. The approach utilizes GigaEvo, an open-source framework for quality-diversity search, where a frozen LLM acts as a mutation operator to rewrite candidate artifacts (programs, policies, or prompts) based on task-specific fitness functions.

The framework was evaluated across three distinct clinical reasoning settings:

1. Urgency Triage: Optimizing the assignment of urgency levels (Emergency, Non-Emergency, Self-Care) under asymmetric clinical risks. Candidates are executable Python modules that decompose decision logic, utilize retrieval, and apply voting mechanisms. Fitness functions prioritize emergency recall and safety-weighted accuracy.
2. Interactive Consultation: Optimizing sequential evidence acquisition in multi-turn dialogues. Candidates implement abstention policies to decide when to ask follow-up questions versus when to commit to a final answer. The goal is to improve the accuracy–cost frontier (answer accuracy vs. token usage).
3. Medical Image Classification: Optimizing prompt-only strategies for fixed Vision-Language Models (VLMs). Candidates are Python modules that generate prompts for frozen MedGemma models to classify pediatric chest X-rays (PneumoniaMNIST), enforcing strict JSON output validity.

In all settings, the evolution process selects high-performing candidates across behavioral dimensions (e.g., accuracy vs. recall, or accuracy vs. cost) rather than optimizing for a single global maximum, preserving diverse solution types.

Key Contributions

The paper makes three primary contributions:

1. Demonstration of Efficacy: The authors show that LLM-guided MAP-Elites evolution can generate and refine medical decision strategies that match or outperform manually designed baselines across three distinct decision settings (triage, consultation, and image classification).
2. Comprehensive Evaluation: The study provides quantitative and qualitative analyses of the gains, examining accuracy, safety-relevant behavior (e.g., undertriage rates), interaction costs, cross-model transfer, and held-out generalization.
3. Mechanistic Insight: The authors demonstrate that performance gains arise from interpretable, program-level mechanisms rather than superficial prompt rewording. These include calibrated triage boundaries, targeted evidence acquisition, selective commitment strategies, and finding-oriented visual decision rules.

Key Results

Triage (Semigran & MIMIC-ESI)

• Semigran Benchmark: The best evolved program (SG-c1189) increased accuracy from 77.3% to 87.1% and emergency recall from 0.60 to 0.97, significantly outperforming the best initial program and other published LLM baselines (excluding human physicians). It maintained high self-care recall, avoiding the degenerate solution of over-predicting emergencies.
• MIMIC-ESI Benchmark: The best evolved program (MIMIC-023) achieved 62.0% exact accuracy and 77.0% range accuracy (including safer overtriage), outperforming hand-designed baselines and competing with RAG-assisted systems. It reduced severe undertriage from 3.6% to 1.2% relative to strong baselines.
• Transfer: Evolved strategies transferred effectively to unseen LLM backbones (e.g., Qwen, GPT-oss variants) without code changes. However, specialization on the Semigran set led to conservative overtriage on the independent Levine set, highlighting the trade-off between safety-oriented specialization and robustness.

Interactive Consultation (iMEDQA & iCRAFTMD)

• Evolved policies improved the accuracy–cost frontier across multiple models (Llama-3, Qwen-3.5, Gemma-4).
• On Llama-3-8B, the evolved strategy improved accuracy by +3.1% while reducing token usage by 89.6%.
• On Gemma-4-31B, it achieved a massive 92.6% reduction in token usage with only a marginal accuracy drop (75.6% vs. 76.4% baseline).
• These gains transferred to the held-out iCRAFTMD dataset, confirming that the strategies improve decision quality without simply increasing interaction length.

PneumoniaMNIST (Image Classification)

• Prompt-only evolution improved frozen MedGemma models across all tested resolutions (28x28 to 224x224).
• For MedGemma-27B at 224x224, the evolved prompt achieved 84.46% accuracy and 83.88% macro-F1, surpassing the best initial seed.
• The gains were most pronounced for smaller models (MedGemma-4B) at lower resolutions, where the evolved prompts effectively tuned decision thresholds for ambiguous visual inputs.

Significance and Claims

The paper claims that LLM-guided evolution is a promising, reproducible tool for discovering effective, inspectable, and adaptable medical decision strategies. Unlike fine-tuning, this approach operates at inference time, making it practical for adapting frozen models under explicit metrics and structured-output constraints.

The authors emphasize that the improvements stem from interpretable program-level mechanisms:

• Triage: Recalibrating decision boundaries to prioritize safety (emergency recall) without sacrificing specificity.
• Consultation: Allocating computational budget to uncertainty, using utility-based question selection, and applying self-consistency only at forced commitment steps.
• Image Classification: Transforming fixed prompts into executable visual decision policies that check for specific radiological findings (e.g., air bronchograms, consolidation) rather than relying on generic classification.

The authors maintain a modest stance, noting that these systems are research prototypes and not substitutes for prospective clinical validation, robust calibration, or safety governance. They acknowledge limitations, including the risk of overfitting on small benchmarks (Semigran), the retrospective nature of MIMIC-ESI data, and the non-negligible search costs incurred during inference-time evolution. However, they argue that this method provides a viable alternative for optimizing medical pipelines where manual engineering is inconsistent and fine-tuning is infeasible.