MedUPS: Towards Diagnostic Assistance in Uncommon Medical Cases with Large Language Models

Technical Summary: MedUPS – Diagnostic Assistance in Uncommon Medical Cases with Large Language Models

Problem Statement
Current clinical large language models (CLLMs) and medical benchmarks predominantly evaluate performance based on the final diagnosis derived from a fully specified case vignette. This "exam-style" evaluation fails to reflect the reality of clinical practice, particularly for uncommon and off-guideline cases. In these scenarios, physicians must navigate diagnostic uncertainty by making a series of intermediate management decisions (e.g., ordering specific tests, involving specialists, or pursuing differential diagnoses) based on accumulating, incomplete evidence. Existing benchmarks rarely capture this longitudinal decision-making process, and the final diagnosis is often an unreliable training target for rare conditions due to its scarcity, variability in free-text phrasing, and inherent uncertainty even in published records. Consequently, there is a lack of supervision signals that mirror how clinicians reason forward from partial information toward the next appropriate action.

Methodology
The authors introduce MedUPS, an alignment framework, and MedUPSQA, a dataset designed to supervise models on intermediate clinical decision points rather than terminal diagnoses.

1. Dataset Construction (MedUPSQA):

   • Source: The dataset is derived from 5,535 real-world case reports (extended from the CUPCase corpus) published in the BMC Journal of Medical Case Reports.
   • Segmentation: Free-text case presentations are automatically segmented into chronologically ordered, clinically coherent "chunks" ($c_1, \dots, c_T$) using few-shot prompting with GPT-4o. Each chunk captures a single dominant clinical concept (e.g., a symptom evolution or lab result).
   • Task Formulation: For each prefix of the trajectory $\{c_1, \dots, c_i\}$, the model is tasked with predicting the next clinically relevant step ($a_i$) that occurs in the subsequent chunk ($c_{i+1}$).
   • Generation & Validation: An external LLM (DeepSeek-R1) generates questions ($q_i$), chain-of-thought reasoning ($r_i$), and answers ($a_i$) based on the context. A separate LLM-as-a-Judge (DeepSeek-Chat) validates these instances, ensuring the generated answer is clinically equivalent to the reference in $c_{i+1}$ based on a four-criterion rubric: Correctness, Specificity, Groundedness, and Completeness.
   • Scale: The final dataset contains 21,874 validated next-step decision points.

2. Alignment Framework (MedUPS):

   • Objective: The framework optimizes models to predict the next step using Group Relative Policy Optimization (GRPO). This is a critic-free policy-gradient method that normalizes rewards against a group mean, avoiding the need for a separately trained value network.
   • Reward Signal: The reward is provided by an external LLM judge scoring the model's output against the rubric. The score is a weighted sum of the four criteria, normalized to $[0, 1]$.
   • Optimization: Models are fine-tuned using Low-Rank Adapters (LoRA) with a frozen reference model for KL regularization. The process involves sampling groups of completions, scoring them, and updating the policy to favor completions exceeding the group average.
   • Baselines: The authors compare GRPO against Supervised Fine-Tuning (SFT) on the identical mid-stream pairs to isolate the contribution of the optimizer from the target. They also evaluate unaligned open models and frontier closed models (e.g., GPT-5.6 variants) as baselines.

Key Contributions

1. Mid-stream Alignment Objective: The paper formalizes next-step prediction along patient timelines as a reinforcement learning objective. It demonstrates that GRPO on this single objective improves clinical reasoning on uncommon cases across three unrelated backbones (Qwen3.6-27B, Qwen3.5-9B, HuatuoGPT-3-8B) without requiring final diagnostic labels.
2. MedUPSQA Dataset: An open dataset of 21,874 validated next-step decision points with reasoning traces, derived from segmented real-world case reports, alongside aligned model checkpoints and code.
3. Alignment over Scale: The study provides evidence that targeted intermediate supervision can be more impactful than model size for uncommon-case reasoning. Smaller, mid-stream-aligned open models match or exceed substantially larger open and frontier models on next-step prediction tasks.

Results

• Performance Gains: Mid-stream alignment significantly improved next-step accuracy across all tested backbones:
  • Qwen3.6-27B: Increased from 55.2% to 66.7%.
  • Qwen3.5-9B: Increased from 47.2% to 57.8%.
  • HuatuoGPT-3-8B: Increased from 37.8% to 44.4%.
  • All improvements were statistically significant (95% CI).
• Comparison to SFT: While SFT on the same data improved all backbones above their base versions, GRPO outperformed SFT on the largest backbone (Qwen3.6-27B), suggesting that group-relative RL is effective when the base policy is strong enough to generate informative variations.
• Scale vs. Alignment: The aligned 27B open model (66.7%) surpassed unaligned frontier models (GPT-5.6 variants, ~60%) and unaligned larger open models (Qwen3.5-397B-A17B, ~53%). The aligned 9B model also outperformed the unaligned 27B base.
• Judge Robustness: The authors acknowledge that the evaluation relies on an LLM-as-a-Judge. Re-scoring with different judges (GPT-5.6, Opus 5) showed moderate agreement (Cohen's $\kappa \approx 0.56-0.65$) and a drop in absolute accuracy (from 67.2% to 58.0% with GPT-5.6), indicating that while the relative ordering of aligned vs. base models holds, the absolute scores and the margin over frontier models are judge-dependent.
• Error Analysis: Analysis of failures revealed that many "incorrect" predictions were not clinical errors but rather disagreements about the specific order of defensible steps or the granularity of the reference report. Genuine clinical reasoning errors (Category E) accounted for only ~2.0% of decisions.

Significance and Claims
The paper claims that for uncommon medical cases, the training objective (mid-stream alignment) matters more than model scale. By shifting the focus from final diagnosis to the "next appropriate action," the framework better aligns with the clinical reality of reasoning under uncertainty. The authors posit that this approach offers a viable path for deploying clinically useful systems on-premises using open, smaller models, thereby addressing privacy and cost constraints associated with closed frontier APIs.

The authors remain modest about their claims, explicitly noting:

• The evaluation relies on LLM judges, not clinician adjudication, so the results reflect alignment with the judge's criteria rather than absolute clinical truth.
• The superiority of aligned open models over frontier models is specific to the evaluation judge used and requires further validation with independent judge families.
• The dataset construction is fully automated without clinician verification of the chunking or question generation, which is treated as a principal limitation.
• The work is intended as clinical decision support under clinician oversight, not as a replacement for human judgment.