💻 computer science

Ask, Condition or Abstain: Reinforcement Learning for Missing-Premise Reasoning

This paper introduces ACA-RL, a reinforcement learning framework and the Missing-Premise Benchmark (MPB) that enable reasoning models to effectively handle underdetermined queries by learning to ask for missing information, condition answers on unknowns, or abstain, rather than forcing incorrect responses.

Original authors: Yongqi Tong, Zhenyu Zhang, Zimi Liu, Kewei Fu, Mingli Song, Haofei Zhang, Junshao Zhang, Hong Zhu, Jiang-Ming Yang, Xin Zhang, Jianshe Li

Published 2026-08-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yongqi Tong, Zhenyu Zhang, Zimi Liu, Kewei Fu, Mingli Song, Haofei Zhang, Junshao Zhang, Hong Zhu, Jiang-Ming Yang, Xin Zhang, Jianshe Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are asking a question to a very knowledgeable assistant, but you have accidentally left out a crucial piece of information needed to solve the puzzle. Perhaps you ask for the total cost of a trip but forget to mention the destination, or you ask for a solution to a math problem but omit a specific number. In the real world, a helpful human would pause, notice the gap, and ask for the missing detail. They would not guess the destination or invent a number just to give you an answer. For a long time, artificial intelligence systems trained to solve problems have struggled with this specific situation. When faced with an incomplete question, these systems often feel compelled to fill in the blanks themselves, confidently producing a wrong answer or a made-up fact. This behavior, known as hallucination, happens because the models are trained primarily to find a final answer, not to recognize when an answer is impossible to find.

Researchers have been working to teach these systems to recognize uncertainty, but simply telling them to say "I don't know" is not always the most helpful response. Sometimes, the best reaction is to ask the user for the missing piece of information. Other times, the system can provide a useful answer that depends on a variable, such as saying, "The cost will be ten dollars if the trip is to Paris, but twenty dollars if it is to London." This paper introduces a new method to train artificial intelligence to handle these incomplete questions with the same care a human would. The researchers developed a system that teaches models to either ask for the missing information, provide a conditional answer, or politely refuse to guess when no useful response is possible.

The team, led by researchers at Ant International and Zhejiang University, created a new training approach called Ask-Condition-Abstain Reinforcement Learning. To teach the models this behavior, they first had to create a massive library of practice problems that were intentionally broken. They took 120,000 well-posed, solvable problems and carefully removed or altered a single critical piece of information from each one. This process was not random; they used a structured map of how the problem should be solved to identify exactly which fact was necessary for the answer. By surgically removing these facts, they created a dataset where the only correct response was to admit the question was incomplete.

Once this training data was ready, they taught the models using a system of rewards that encouraged specific behaviors. Instead of just rewarding a correct final answer, the system gave points for asking clarifying questions or explaining what was missing. It gave fewer points for simply refusing to answer, and it gave negative points for making up facts or guessing. The goal was to shift the model's instinct from "guess the answer" to "identify the gap." To test if this training actually worked, the researchers built a new benchmark called the Missing-Premise Benchmark. This test set contained 274 carefully verified problems where information was missing, covering math, logic, and real-world word problems.

The results showed that the new training method significantly improved how the models handled incomplete questions. On the new benchmark, the models trained with this method scored much higher than those trained only to find answers or those trained simply to say "I don't know." The trained models were far less likely to hallucinate or invent facts. Instead, they frequently chose to ask for the missing information or to explain how the answer would change depending on the unknown variable. Crucially, this improvement did not come at the cost of their ability to solve normal, complete problems. When tested on standard math and logic puzzles where all information was provided, the models performed just as well as before, showing that learning to recognize uncertainty did not make them worse at solving solvable tasks.

The study suggests that the next step for intelligent systems is not just to be faster or more accurate at solving problems, but to be better at knowing when a problem cannot be solved as stated. By teaching models to expose missing information rather than guessing, this approach supports the creation of safer and more reliable assistants. These systems can decide when to ask a user for help or when to use a tool to find a missing fact, rather than confidently providing a wrong answer. The researchers note that while their method works well for logical gaps in structured problems, real-world questions can be messier and more ambiguous. However, the work provides a clear path forward for teaching artificial intelligence to recognize the limits of its own knowledge and respond constructively when information is incomplete.

Technical Summary: Ask, Condition or Abstain: Reinforcement Learning for Missing-Premise Reasoning

1. Problem Statement

Current reasoning models are predominantly optimized via Reinforcement Learning (RL) on fully specified problems where a single, verifiable final answer exists. However, realistic user queries often omit critical premises, leaving the problem underdetermined. In these scenarios, models trained solely for definite answers tend to hallucinate confident solutions even when the question does not determine one.

The paper argues that the ideal response space for missing-premise problems extends beyond simple refusal ("I don't know"). A robust model should be capable of three distinct behaviors:

Ask: Proactively requesting the missing premise.
Condition: Formulating an answer conditional on the unknown variable.
Abstain: Refusing to answer when no informative conditional response is possible.

Existing approaches, such as retrieval or inference-time screening, do not teach the base model to intrinsically recognize underdetermined prompts. Furthermore, simple prompting (e.g., "LM Introspection") is insufficient to reliably induce these specific response policies.

2. Methodology: ACA-RL

The authors propose Ask-Condition-Abstain Reinforcement Learning (ACA-RL), a data-augmented RL framework designed to learn these response policies.

2.1 Data Construction: Reasoning-Graph-Guided Synthesis

To train on missing-premise instances, the authors developed a pipeline to synthesize 120,000 training examples from well-posed problems. This process ensures that the "missing" information is a localized logical gap rather than random noise. The pipeline involves:

Deconstruction & Graph Generation: Parsing a well-posed problem into a Directed Acyclic Graph (DAG) representing the reasoning path and dependencies between conditions and the answer.
Surgical Condition Perturbation: Identifying a critical condition on the solution path and applying one of six perturbation strategies (e.g., removing a numerical value, unquantifying a relationship, or disrupting an entity) to create an underspecified variant ( $s'$ ).
Rewrite & Recheck: Reconstructing the problem into a fluent word problem and filtering out samples where the gap is not identifiable or the problem remains solvable.

2.2 Structured Reward Function

Unlike standard answer-only RL which rewards only correct final answers, ACA-RL employs a structured reward function ( $R_{ACA}$ ) based on a behavioral taxonomy of five categories:

Silent Hallucination (SH): Fabricating an answer without acknowledgment (Reward: -1.0).
Explicit Assumption (EA): Giving a definite answer but stating the assumption (Reward: -0.3).
Abstention (Abs): Correctly identifying the problem as underspecified and refusing (Reward: 0.3).
Conditional Formulation (Cond): Expressing the answer as a formula dependent on the missing variable (Reward: 0.6).
Active Elicitation (Elicit): Proactively asking for the missing premise (Reward: 1.0).

The objective is to maximize the expected value of these behaviors, shifting the policy away from hallucination and generic refusal toward more informative interactions.

2.3 Evaluation: Missing-Premise Benchmark (MPB)

The authors introduce the Missing-Premise Benchmark (MPB), a human-verified dataset of 274 instances spanning mathematical, logical, and real-world word problems. MPB evaluates models across the same five behavioral categories, reporting a "Behavior Score" (a weighted average of the categorical rewards) rather than just accuracy or refusal rates.

3. Key Results

Experiments were conducted on Qwen3 (8B, 14B) and Llama3.1-8B models, comparing ACA-RL against Cold-start SFT, Vanilla PPO (answer-only), and IDK-RL (trained to refuse).

Missing-Premise Robustness: ACA-RL significantly outperforms baselines on MPB. On Qwen3-8B, ACA-RL achieved a Behavior Score of 51.73, compared to 8.66 for Vanilla PPO and 48.72 for IDK-RL. This indicates that ACA-RL successfully learns to ask or condition rather than just refuse or hallucinate.
General Reasoning Preservation: Crucially, ACA-RL maintains competitive performance on standard well-posed benchmarks (GSM8K, MATH-500, AIME'24). For instance, on Qwen3-8B, the drop in GSM8K performance relative to Vanilla PPO was smaller for ACA-RL (-2.35) than for IDK-RL (-5.83).
Behavioral Shift: Analysis shows that while IDK-RL maximizes refusal rates, ACA-RL shifts the distribution toward Conditional Formulation and Active Elicitation. The "Ask" and "Condition" behaviors emerge more gradually than "Abstain," suggesting a more complex policy learning process.
Data Efficiency: Ablation studies confirm that the reasoning-graph-guided synthesis is superior to other data sources (e.g., Treecut or SUM-derived data) for this task. A 30% mixture of missing-premise data in the training set provided the best trade-off between robustness and general reasoning capability.

4. Significance and Claims

The paper claims to reframe progress in reasoning models: success should not be measured solely by accuracy on fully specified tasks, but also by the ability to recognize when a task is underdetermined and respond constructively.

Beyond Refusal: The work demonstrates that models can be trained to do more than just say "I don't know"; they can provide conditional answers or solicit missing information, which is more helpful in tutoring systems and agentic workflows.
Evaluation Gap: By introducing MPB, the authors highlight a gap in current NLP evaluation, which often fails to measure how models handle uncertainty and under-specified inputs.
Limitations: The authors modestly note that their training focuses on single-turn responses. The method does not currently train multi-turn clarification policies or tool use (retrieval) to actually acquire the missing information, leaving a gap between identifying the gap and resolving it in a full agent workflow. Additionally, the synthetic data, while controlled, may not fully capture the semantic ambiguity of organic user queries.

In conclusion, ACA-RL provides a concrete framework for training models to navigate uncertainty by asking, conditioning, or abstaining, supported by a new benchmark and a scalable data synthesis pipeline.