Technical Summary: Quantifying and Mitigating Premature Closure in Frontier LLMs

Problem Definition

The paper addresses a critical, underexamined failure mode in Large Language Models (LLMs) applied to medicine: premature closure. In clinical practice, premature closure is the cognitive error of committing to a diagnosis or plan before sufficient information is gathered. The authors define LLM premature closure as "inappropriate commitment under uncertainty": providing an answer, specific clinical guidance, or complying with unsafe instructions when the available information does not support a safe response.

The core problem is that current medical AI benchmarks often assume a correct answer exists and reward its selection, failing to test whether a model can recognize when no option is appropriate. This creates a safety risk where models appear helpful but generate confident, specific, and potentially harmful guidance in scenarios requiring clarification, abstention, escalation, or refusal.

Methodology

The study evaluated five frontier LLMs (GPT-5.4, Claude Opus 4.7, Grok 3, Gemini 2.5 Pro, and DeepSeek R1) across structured and open-ended medical tasks under two conditions: a baseline prompt and a safety-oriented prompt.

1. Structured Multiple-Choice Evaluation

The authors modified two standard benchmarks, MedQA (USMLE-style) and AfriMed-QA (Pan-African context), to include "unanswerable" items.

• INTACT items: Retained the correct answer option.
• NOTA (None-of-the-Above) items: The correct option was removed entirely, leaving only incorrect choices.
• Metric: False Action Rate, defined as the proportion of NOTA items where the model selected an answer instead of abstaining.
• Prompting: The safety prompt explicitly instructed models to abstain if no option was clearly correct.

2. Open-Ended Evaluation

The authors utilized HealthBench (861 questions) and HealthBench Professional (191 adversarial red-team queries).

• HealthBench: Included underspecified low-risk, underspecified urgent, sufficient-context, and uncertainty-sensitive cases.
• Adversarial Queries: Designed to push models into overconfident or unsafe answers (e.g., instructing a model to discharge a patient with signs of acute appendicitis).
• Metrics:
  • Premature Closure Rate: Binary labeling of whether the model gave specific guidance when abstention/clarification/escalation was safer.
  • Rubric Scores: Physician-authored criteria scoring alignment with appropriate behavior (ranging from negative penalties for unsafe guidance to positive credit for appropriate actions).
• Prompting: The safety prompt instructed models to express uncertainty, seek clarification, or recommend immediate care if the situation was urgent, rather than speculating.

3. Evaluation Protocol

Responses were evaluated using LLM judges (GPT-5.4 and Claude Opus 4.7) against physician-authored rubrics. Statistical significance was tested using McNemar's test for paired binary outcomes and Wilcoxon signed-rank tests for rubric scores.

Key Results

Structured Tasks (MedQA & AfriMed-QA)

• Baseline Failure: Models frequently selected answers on unanswerable NOTA items. Baseline false action rates were high: 55–81% on MedQA and 53–82% on AfriMed-QA.
• Impact of Safety Prompting: Safety prompting reduced false action rates significantly (averaging a drop from ~70% to ~48%). However, residual failure persisted, with rates remaining substantial (e.g., Claude Opus 4.7 remained at 65–66%).
• Trade-offs: For most models, reducing false action did not significantly impair accuracy on answerable (INTACT) items (average decrease of only 1.0 percentage point). However, Grok 3 showed a trade-off, with reduced false action accompanied by lower accuracy on valid items.

Open-Ended Tasks (HealthBench & Adversarial)

• Baseline Performance: Premature closure was common, particularly in underspecified low-risk scenarios (baseline rates 73–96% across models). Models performed better on urgent cases where risk cues were explicit.
• Adversarial Vulnerability: On adversarial queries, baseline premature closure was severe. Four of five models committed to unsafe guidance on >70% of cases. Notably, Grok 3 and DeepSeek R1 had negative baseline rubric scores (−10.3% and −8.4%), indicating their responses were penalized more heavily for unsafe behavior than credited for any appropriate elements.
• Mitigation Limits: Safety prompting reduced premature closure on adversarial queries by 9–20 percentage points and improved rubric scores (e.g., Grok 3 improved from −10.3% to +4.9%). However, every model still committed prematurely on the majority of adversarial queries even with safety prompting. The lowest residual rate was 50.8% (Claude Opus 4.7).

Significance and Claims

The paper argues that the tendency of frontier LLMs to answer when they should not is a persistent failure mode that transcends evaluation formats. The authors make the following specific claims regarding the significance of their findings:

1. Limitations of Prompting: While safety-oriented prompting is a useful, low-cost, and immediately deployable lever that reduces premature closure, it is not a definitive fix. Substantial residual failure remains, particularly in adversarial and ambiguous scenarios.
2. Selective Sensitivity: Models appear selectively sensitive to explicit urgency cues (escalating when danger is obvious) but remain insensitive to missing context in seemingly benign scenarios. This suggests a gap in "meta-reasoning"—the ability to recognize when reasoning is impossible due to insufficient information.
3. Evaluation Gaps: Current benchmarks emphasizing answer correctness fail to assess uncertainty handling. The authors call for evaluation frameworks that explicitly reward appropriate abstention and incorporate underspecified and adversarial scenarios.
4. Systemic Requirements: Addressing premature closure requires more than prompt engineering. It necessitates system-level design changes, including training objectives that explicitly reward abstention, improved uncertainty calibration, and mechanisms for clarification or escalation.
5. Clinical Implication: For clinicians and health systems, the findings highlight that models performing well on structured benchmarks may still provide inappropriate guidance when faced with incomplete or ambiguous clinical information. Safety prompting should be viewed as a baseline guardrail, not a primary safety mechanism for high-risk applications.

In conclusion, the paper asserts that "knowing when not to answer" is a critical capability that current frontier LLMs do not yet reliably exhibit, and achieving it will require rethinking how models are trained, evaluated, and deployed.