Technical Summary: MedAction: Towards Active Multi-turn Clinical Diagnostic LLMs

1. Problem Formulation

Current Large Language Models (LLMs) in medicine are predominantly evaluated on static, single-turn benchmarks (e.g., MedQA, MedMCQA) where complete patient records—including labs, imaging, and vitals—are provided upfront. This setup oversimplifies real-world clinical practice, where diagnosis is an active, iterative process. Clinicians begin with incomplete information (chief complaint and physical exam), iteratively order tests to resolve uncertainty, interpret results, and progressively narrow the differential diagnosis (DDx) across multiple turns.

The authors identify a critical gap: existing models are trained to reason from complete information but lack the capability to act under evolving, partial evidence. A recent study (Qiu et al., 2025) highlights a sharp performance degradation when state-of-the-art models, including agentic and specialist models, are evaluated on active diagnosis tasks. Through systematic analysis, the paper identifies three recurring failure modes in current LLMs:

1. Ungrounded Test Ordering: Models order tests without a clear connection to their stated hypotheses, lacking case-specific clinical reasoning.
2. Unreliable Diagnostic Update: Models fail to appropriately update their differential diagnosis based on new evidence, either adhering rigidly to an initial hypothesis despite contradictory data or shifting erratically between unrelated diagnoses.
3. Degraded Multi-turn Coherence: As interactions extend, models lose track of ordered tests, obtained results, and unresolved uncertainties, leading to inconsistencies in longer contexts.

2. Methodology: The MedAction Pipeline

To address these deficits, the authors introduce MedAction, a framework for automatically constructing high-quality, multi-turn active diagnosis training data from publicly available medical literature (PMC case reports). The pipeline consists of three stages:

Stage 1: Clinical Environment Construction

Raw case reports are transformed into interactive clinical environments ($E$). An LLM extracts:

• Initial Observation ($o_1$): Chief complaint, history, and physical exam.
• Test Menu ($M$): The set of documented tests available in the case.
• Oracle Function ($f$): A function that returns documented results for ordered tests or marks them as UNAVAILABLE if not present in the source text, simulating partial observability.

Stage 2: Multi-turn Chain-of-Reasoning-Action Trajectory Distillation

State-of-the-art teacher LLMs interact with these environments to generate diagnostic trajectories. Unlike standard Chain-of-Thought (CoT) which assumes complete information, MedAction enforces an explicit Reason-Action pattern at each turn:

1. Analyze: Generate a ranked differential diagnosis ($b_t$) with confidence estimates.
2. Identify: State the key remaining uncertainty ($p_t$) distinguishing top hypotheses.
3. Select: Choose a test ($a_t$) justified by the uncertainty.
4. Observe: Receive the result ($r_t$) from the oracle.

To maximize diversity, the pipeline employs tree-structured sampling, launching branched rollouts from intermediate states of existing trajectories to capture multiple valid diagnostic strategies from the same evidence state.

Stage 3: Knowledge-Graph-Grounded Filtering

To distill high-quality data, the authors propose two metrics grounded in medical knowledge graphs (PrimeKG and ClinGraph) to filter trajectories:

• Disease Trajectory Consistency (DTC): Measures the shortest-path distance on PrimeKG between the model's current top hypothesis and the ground-truth diagnosis. Trajectories are retained if they show convergence (decreasing DTC) toward the correct answer. Partially correct trajectories that eventually converge are kept, while divergent ones are pruned.
• Reasoning–Action Consistency (RAC): Measures whether ordered tests drove meaningful changes in the differential diagnosis on ClinGraph. It verifies that belief updates are evidence-driven. Turns where hypothesis shifts are unrelated to gathered evidence are removed.

This process yields MedAction-32K, a dataset of 32,681 trajectories derived from 2,896 PMC cases.

3. Key Contributions

1. New Formulation and Failure Analysis: The paper reformulates clinical diagnosis as a multi-turn active setting with iterative test ordering and identifies three specific failure modes (ungrounded ordering, unreliable updates, and coherence degradation) that explain current model limitations.
2. Dataset Construction and Metrics: The authors introduce a tree-structured distillation pipeline and release MedAction-32K (training set) and MedAction-300 Hard (a test set of 300 rare-disease cases requiring longer trajectories). They also propose DTC and RAC metrics for rigorous data filtering and evaluation.
3. Efficient and Effective Model: Fine-tuning an 8B-parameter model (II-Medical-8B) on MedAction-32K achieves state-of-the-art performance among open-source models on active diagnosis benchmarks, demonstrating that high-quality trajectory data drives larger gains than model scale alone.

4. Experimental Results

The authors evaluated their 8B model (II-Medical-8B SFT) against open-source general models (e.g., Qwen-QwQ 32B), closed-source models (e.g., GPT-5.4, OpenAI-o3-mini), and medical domain models (e.g., MedGemma 27B, Baichuan-M3 235B).

• Performance: On the MedR-Bench and the curated MedAction-300-Hard benchmarks, the MedAction-trained 8B model achieved the highest diagnosis accuracy among open-source models, outperforming the 235B-parameter Baichuan-M3 teacher.
• Test Recommendation: The model showed consistently high precision and recall in test ordering, addressing the "ungrounded test ordering" failure mode observed in the base model.
• Ablation Studies:
  • Gold vs. Distilled: Training on "Gold" trajectories (reconstructed directly from case reports) alone failed to improve reasoning; environment-interactive distillation was essential.
  • Format Flexibility: Mixing a small proportion (10%) of free-form trajectories prevented overfitting to rigid output schemas and improved test recommendation quality.
  • Diversity: Multi-teacher diversity (combining GPT and Baichuan teachers) and multi-branch sampling significantly improved performance, provided low-quality trajectories were filtered.
  • Filtering Strategy: The proposed DTC+RAC filtering outperformed simple "correctness-only" filtering. It recovered useful intermediate reasoning from partially correct trajectories and reduced ungrounded actions, leading to better data efficiency and scaling properties.

5. Significance and Claims

The paper claims that the core deficit in current medical LLMs is a lack of training on acting under partial evidence. By shifting from static QA to active, multi-turn diagnostic simulation, MedAction equips models with the ability to sequentially decide which information to gather and how to update hypotheses.

The authors emphasize that data quality and structure are more critical than sheer model scale for this task. Their 8B model, trained on carefully distilled and filtered multi-turn trajectories, surpasses significantly larger models (including 235B and 27B parameter models) and approaches the performance of top-tier closed-source models. The work provides a foundational framework for developing clinical decision-support tools that genuinely reflect the iterative nature of medical diagnosis, with potential for extension into reinforcement learning post-training approaches.