Technical Summary: Reasoning While Asking (PIR)

Problem Statement

Current reasoning-oriented Large Language Models (LLMs), such as GPT-o1 and DeepSeek-R1, operate under a "blind self-thinking" paradigm. While these models excel at generating internal Chain-of-Thought (CoT) traces, they fundamentally lack the ability to recognize when their reasoning is underinformed or based on ambiguous user instructions. When faced with incomplete or ambiguous prompts—a common occurrence in real-world interactions—these models tend to persist with extensive internal reasoning regardless of missing premises. This behavior leads to "overthinking," hallucinations, misaligned conclusions, and significant computational waste. Furthermore, existing interactive LLMs often remain "passive solvers," reacting to user feedback only after errors occur rather than proactively reducing ambiguity during the reasoning process.

Methodology: Proactive Interactive Reasoning (PIR)

The authors propose Proactive Interactive Reasoning (PIR), a paradigm that transforms LLMs from passive solvers into proactive inquirers. PIR interleaves reasoning with clarification, allowing models to detect missing premises and autonomously initiate interaction with the user. The framework is implemented through two distinct phases:

Phase I: Interactive Capability Activation (Uncertainty-Aware SFT)

To equip models with the structural ability to ask questions, the authors introduce an uncertainty-aware supervised fine-tuning (SFT) procedure.

• Uncertainty Detection: The system computes the Predictive Entropy (PE) of reasoning steps. Steps with high PE (indicating low model confidence) are identified as critical decision points.
• Data Augmentation: At these high-uncertainty junctures, the linear CoT traces are converted into an interleaved "think-ask-respond" format. A strong instruction-following model (e.g., GPT-4o-mini) inserts a clarification question and simulates a corresponding user response.
• Training: The model is fine-tuned on these augmented trajectories to learn the causal pattern: uncertainty → clarification → improved reasoning.

Phase II: User-Intent Alignment (US-GRPO)

To optimize when and how to ask efficiently, the authors develop User-Simulator Group Relative Policy Optimization (US-GRPO).

• Dynamic User Simulator: A user simulator $S$, conditioned on a specific user intent $I$, generates responses consistent with the underlying goal. This creates a controllable environment for training without the cost of real human interaction.
• Composite Reward Function: The optimization is driven by a reward function $R(y)$ that combines:
  1. Output Reward ($R_{output}$): Ensures the final answer matches the ground truth.
  2. Reasoning Reward ($R_{reason}$): Evaluated only when the answer is correct, this component balances Efficiency (penalizing excessive interaction turns) and Helpfulness (evaluating the quality of clarification questions via an LLM judge).
• Optimization: Using GRPO, the model learns to prioritize intent resolution over ungrounded self-thinking, minimizing unnecessary turns while maximizing task success.

Key Contributions

1. Formalization of "Blind Self-Thinking": The paper identifies and formalizes the limitation where reasoning models fail to recognize missing information, leading to hallucinations and inefficiency.
2. PIR Framework: A novel two-stage framework (Uncertainty-Aware SFT + US-GRPO) that enables models to proactively detect uncertainty and initiate clarification.
3. US-GRPO Algorithm: A reinforcement learning method incorporating a dynamic user simulator and a composite reward system (balancing accuracy, helpfulness, and efficiency) to align reasoning behavior with user intent.
4. Generalization: The approach demonstrates robustness not only in interactive settings but also in non-interactive benchmarks involving missing premises.

Experimental Results

Extensive experiments were conducted on mathematical reasoning (Math-Chat), code generation (BigCodeBench-Chat), and document editing (DocEdit-Chat).

• Performance Gains: PIR consistently outperformed strong baselines, including standard instruction-tuned models, proactive prompting, and collaborative frameworks (CollabLLM).
  • Accuracy: Improved by up to 32.70% on Math-Chat (from 22.90% to 32.70%).
  • Pass Rate: Improved by 22.90% on BigCodeBench-Chat.
  • BLEU Score: Improved by 41.36 on DocEdit-Chat.
• Efficiency: PIR reduced reasoning computation by approximately 2,000 tokens per task on average and cut unnecessary interaction turns by nearly 50%.
• Reliability: In "Missing Premise" tests (MIP), PIR remained highly robust, whereas base models and GPT-4o-mini suffered severe degradation due to hallucinating missing conditions. PIR successfully avoided "blind overthinking" by seeking clarification first.
• Human Evaluation: In a double-blind user study on the GSM8K Missing Premise dataset, users preferred PIR over reasoning-oriented LLMs 52.45% of the time, citing higher accuracy and significantly lower interaction latency.

Significance and Claims

The paper claims that PIR establishes a new foundation for reliable and human-centric intelligent systems by bridging the gap between internal computation and external verification. By transforming LLMs into proactive inquirers, the framework addresses the critical flaw of "blind self-thinking," ensuring that reasoning is grounded in verified premises rather than assumptions. The authors assert that this approach offers a principled way to align reasoning with user intent under ambiguity, achieving superior problem-solving accuracy while maintaining high computational efficiency. The work suggests that future reasoning models should integrate proactive clarification mechanisms to handle the inherent uncertainty of real-world user interactions effectively.