Technical Summary: Conformal Thinking: Risk Control for Reasoning on a Compute Budget

1. Problem Statement

Reasoning Large Language Models (LLMs) enable "test-time scaling," where performance improves as the token budget increases. However, this creates a practical challenge: determining when to stop reasoning. Existing adaptive reasoning mechanisms typically rely on a single upper threshold of confidence (or uncertainty) to halt generation once the model appears confident. This approach faces two critical inefficiencies:

1. Over-thinking: Models may continue reasoning after a correct answer is already available, wasting compute.
2. Under-thinking on hard instances: On difficult or unsolvable problems, the model's confidence may never reach the upper threshold, causing the system to exhaust the maximum token budget without success.

Furthermore, setting the specific threshold value is a practical challenge. The relationship between a threshold value and the resulting error rate is often uninterpretable, signal-dependent, and varies across tasks. Current methods often rely on hand-tuned cutoffs or heuristic sweeps, which lack theoretical guarantees regarding the actual error rate (risk) incurred.

The authors reframe the problem of setting a reasoning budget not as a hyperparameter tuning exercise, but as a risk control problem. The goal is to limit the error rate to a user-specified tolerance ($\epsilon$) while minimizing computational cost.

2. Methodology: Conformal Thinking

The proposed framework, Conformal Thinking, introduces a dual-threshold stopping mechanism controlled by distribution-free risk control algorithms.

2.1 Dual-Threshold Stopping Policy

Instead of a single upper threshold, the method employs two thresholds to manage two distinct types of risk:

• Upper Threshold ($\lambda_+$): Controls False Positive Risk. It stops reasoning when the model is sufficiently confident that it has found the correct answer. This prevents wasting tokens on solvable instances where the answer is already known.
• Lower Threshold ($\lambda_-$): Controls False Negative Risk. It stops reasoning when the model's confidence is not increasing at a sufficient rate, indicating the instance is likely unsolvable within the remaining budget. This prevents wasting tokens on hopeless instances.

The stopping time $\tau$ is defined as:
$$\tau = \min \{ t \ge 1 : \tilde{s}_t \ge \lambda_+ \lor \tilde{s}_t \le \lambda_- \}$$
where $\tilde{s}_t$ is a stabilized uncertainty signal (e.g., confidence or entropy) at step $t$.

2.2 Dynamic Lower Threshold

A key innovation is the parametric lower threshold. Unlike the static upper threshold, the lower threshold is a dynamic function of the reasoning steps (or token usage). It enforces a "schedule" of progress: the model must demonstrate a regular increase in confidence to continue.
The lower threshold is modeled as a sigmoid function:
$$\lambda_-(t; c, s, l, u) = \sigma(c (\omega_t - sB), l, u)$$
where $\omega_t$ is the cumulative tokens generated, $B$ is the total budget, and parameters $c, s, l, u$ control the slope, shift, and bounds of the confidence requirement. If the model's confidence trajectory fails to meet this dynamic schedule, reasoning is preemptively halted.

2.3 Loss Functions

The framework defines specific loss functions to quantify the trade-offs:

• Predictive Losses:
  • $\ell_+$: Measures false positives (stopping early on an incorrect answer).
  • $\ell_-$: Measures false negatives (stopping on an instance that could have been solved later). This is "farsighted," checking if a correct answer exists at any future step $k \in (t, T]$.
• Efficiency Losses:
  • $J_+$: Measures wasted tokens after the "oracle" stopping time (when the answer first became correct).
  • $J_-$: Measures wasted tokens on instances that were ultimately unsolvable.

2.4 Risk Control via UCB

To select the thresholds ($\lambda_+$) and dynamic parameters ($c$) without overfitting to a finite validation set, the authors employ distribution-free risk control using the Upper Confidence Bound (UCB) algorithm (Bates et al., 2021).

1. Calibration: On a held-out validation set, the algorithm computes an adjusted risk estimate that includes a finite-sample correction term.
2. Selection: It selects the signal and threshold pair that satisfies the user-specified risk tolerance ($\epsilon$) with high probability ($1-\delta$) while minimizing the estimated efficiency loss.
3. Two-Step Procedure: The method decouples the selection of the upper and lower thresholds to simplify optimization, first fixing the upper threshold to control false positives, then selecting the lower threshold parameters to control false negatives.

3. Key Contributions

1. Risk-Control Framework: A principled approach to setting stopping rules for adaptive reasoning that maps user-specified risk tolerances directly to stopping criteria, replacing opaque threshold tuning.
2. Dual-Threshold Mechanism: The introduction of a novel parametric lower threshold that halts reasoning on unsolvable instances, complementing the standard upper threshold.
3. Loss Function Definition: A collection of loss functions capturing both predictive errors (false positives/negatives) and efficiency costs (wasted tokens) specific to early-stopping in reasoning LLMs.
4. Empirical Validation: Demonstration that this approach achieves user-specified risk targets with high probability and improves computational efficiency compared to single-threshold or naive baseline methods.

4. Experimental Results

The authors evaluated the framework on diverse reasoning tasks (AIME, DeepScaleR, GPQA-Diamond, MathVision) using models like Qwen3-8B/30B and DeepSeek-R1-Distill.

• Risk Control Validity: Experiments confirmed that the UCB-based selection ensures the realized test risk stays below the user-specified target $\epsilon$ with high probability. In contrast, naive cross-validation frequently violated risk targets, particularly for false negative risks controlled by the lower threshold.
• Efficiency Gains:
  • Ensembling Signals: The framework automatically selects the most efficient uncertainty signal (e.g., confidence, probe, EAT) for a given risk level, outperforming single-signal baselines.
  • Lower Threshold Impact: The lower threshold provided significant token savings, particularly on datasets with a high proportion of unsolvable instances. In scenarios where unsolvable instances were common (1:1 or 1:3 ratios), the "Upper-Only" approach failed to save tokens (running to the max budget), whereas the dual-threshold approach shifted the accuracy-compute curve significantly leftward.
• Robustness: The method remained robust under distribution shifts (e.g., different dataset difficulties or reasoning lengths), whereas naive methods suffered from severe excess risk.

5. Significance and Claims

The paper claims that Conformal Thinking provides a necessary shift from heuristic threshold tuning to interpretable risk management. By allowing users to specify an acceptable error rate (risk tolerance) rather than an arbitrary confidence value, the framework makes adaptive reasoning more reliable and deployable.

The authors emphasize that their method does not redesign the decoding process or select among completed trajectories; rather, it provides an instance-level stopping rule for ongoing reasoning. The significance lies in its ability to:

1. Guarantee risk bounds in finite-sample settings.
2. Simultaneously address the inefficiencies of over-thinking and under-thinking.
3. Adaptively allocate compute budgets based on instance difficulty while adhering to strict safety/error constraints.

The authors note limitations, including the assumption of monotonicity in risk functions and the current restriction to structured reasoning tasks (math/science) where uncertainty signals are well-defined. Future work is suggested for agentic frameworks and alternative test-time scaling methods.