Technical Summary: Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Information

1. Problem Statement

Large Reasoning Models (LRMs) are increasingly deployed in high-stakes domains such as medical AI, where user queries often lack critical information (e.g., missing symptoms, lab results, or context). A critical failure mode identified in this work is that while models may recognize during intermediate reasoning that a problem is under-specified, they frequently fail to translate this recognition into a final refusal to answer. Instead, they continue reasoning, fabricate missing premises, and produce unsupported definitive answers.

The authors formalize this phenomenon as the detection-to-abstention gap: the discrepancy between a model's ability to detect insufficient information (Detection Rate, DR) and its ability to act on that detection by abstaining (Overall Abstention Rate, OAR). Existing methods, including standard Reinforcement Learning (RL) and prompting, often improve detection but fail to close the gap, leaving models that "detect but do not abstain." This is particularly dangerous in safety-critical settings where an unsupported answer is more harmful than a refusal.

2. Methodology: Judge-Then-Solve (JTS)

To close this gap, the paper proposes Judge-Then-Solve (JTS), a trajectory-level reasoning-control framework. Unlike treating abstention merely as a final-answer style, JTS reframes it as an explicit control decision made before solution generation.

Core Mechanism

JTS decomposes the inference process into two distinct stages:

1. Answerability Judgment: The model must first generate a structured <answerability_judge> block containing:
   • Contextual Audit: An analysis of whether the provided context supports a unique valid answer.
   • Integrity Check: A verification of missing premises or ambiguities.
   • Conclusion: A binary decision of ANSWERABLE or UNANSWERABLE.
2. Conditional Execution:
   • If UNANSWERABLE: The model must immediately terminate reasoning and output an abstention response.
   • If ANSWERABLE: The model proceeds to generate the solution within the same reasoning block.

Training Strategy

JTS is instantiated through a two-stage training process:

1. Supervised Warm-up: The model is fine-tuned on a dataset of 1,091 trajectories formatted with the JTS structure. This stage teaches the structural prior of the framework (the <answerability_judge> block and the immediate termination rule) rather than optimizing abstention directly.
2. Reinforcement Learning (RL) with Structured Rewards: The model is optimized using Group Relative Policy Optimization (GRPO) with a composite reward function $R(y)$:
   • Format Reward ($R_{format}$): Enforces the strict JTS structure (valid tags, conclusion placement).
   • Consistency Reward ($R_{consistency}$): Penalizes contradictions between the judgment (e.g., UNANSWERABLE) and the final output (e.g., providing an answer).
   • Task Reward ($R_{task}$): Rewards correct abstention on under-specified questions and correct answers on well-defined questions; penalizes incorrect abstention on solvable questions.
   • Length-Shaping Reward ($R_{length}$): A conditional signal designed to discourage "overthinking."
     • For under-specified questions where the model fails to abstain, shorter trajectories are rewarded to encourage immediate termination after detection.
     • For well-defined questions where the model fails to answer correctly, longer trajectories are mildly favored to encourage complete reasoning.

3. Key Contributions

1. Formalization of the Detection-to-Abstention Gap: The paper identifies and defines a specific failure mode where models detect missing information but fail to abstain, distinguishing this from simple detection failures.
2. Judge-Then-Solve Framework: A novel reasoning-control paradigm that forces an explicit answerability commitment before solution generation, effectively gating the reasoning trajectory.
3. Conditional Length Shaping: A reward mechanism that specifically targets the "post-detection deliberation" failure, encouraging models to stop reasoning immediately once insufficiency is detected, while preserving reasoning depth for solvable problems.
4. Empirical Validation: Comprehensive evaluation on both dense (DeepSeek-R1-Distill-Qwen-14B) and Mixture-of-Experts (Qwen3-30B-A3B-Thinking) models across multiple benchmarks, including the medical domain (MedIQ).

4. Experimental Results

Experiments were conducted on the Missing-Premise (MIP) dataset and four subsets of AbstentionBench (MMLU-History, MMLU-Math, GPQA-Diamond, MedIQ).

• Closing the Gap: JTS significantly improves Abstention@Detection (A@D), pushing it to near-saturation levels (99.8% for DeepSeek, 99.3% for Qwen). This indicates that once the model detects missing information, it reliably acts on that detection.
• Overall Performance:
  • DeepSeek: OAR increased from 18.6% (Base) to 88.5% (JTS).
  • Qwen: OAR increased from 21.1% (Base) to 72.4% (JTS).
• Efficiency: JTS drastically reduces average response length on under-specified questions (e.g., from ~2600 tokens to ~350 tokens for DeepSeek) by terminating unnecessary deliberation.
• Well-Defined Questions: While JTS introduces a slight trade-off in raw answer rate and correctness on solvable questions, it significantly improves efficiency per token (Correctness/1K tokens), suggesting more concise and effective reasoning.
• Uncertainty Analysis: Token-level entropy analysis reveals that JTS models maintain higher uncertainty (entropy) around missing premises compared to base models, which tend to be overconfident. This uncertainty awareness drives the abstention behavior.
• Secondary Observation: Training on missing-premise data (even with plain RL) was observed to reduce "ineffective self-reflection" and hesitation on difficult but answerable math problems (Omni-Math), leading to slightly improved pass@8 scores in medium-to-hard difficulty ranges.

5. Significance and Claims

The paper claims that reliable abstention is a critical form of reasoning control necessary for the safe deployment of LRMs in high-risk domains. The significance of the work lies in:

• Safety: Demonstrating that models can be trained to refuse unsupported answers in medical and other safety-critical contexts, preventing the generation of harmful hallucinations.
• Mechanism: Showing that the failure to abstain is often a control issue (continuing to reason after detection) rather than a pure capability issue, and that this can be solved by structuring the reasoning trajectory.
• Efficiency: Proving that early termination of unanswerable trajectories reduces computational waste and inference time.

The authors modestly note that while JTS improves abstention, it may lead to a slight increase in false abstentions on well-defined questions, a trade-off they deem acceptable for safety-critical applications. They also acknowledge limitations, including the reliance on LLM-as-a-judge for evaluation and the computational cost of full-parameter RL training.