Technical Summary: Reasoning or Fluency? Dissecting Probabilistic Confidence in Best-of-N Selection

Problem Statement

Recent advancements in Large Language Models (LLMs) have leveraged Best-of-N selection strategies to enhance reasoning capabilities. This approach involves sampling multiple reasoning traces (often via Chain-of-Thought, or CoT) and selecting the optimal one based on a scoring function. While external reward models exist, recent trends favor probabilistic confidence metrics derived directly from the LLM's output token distribution (e.g., self-certainty, log-likelihood, entropy) due to their computational efficiency.

The core assumption underpinning these metrics is that higher probabilistic confidence reflects higher reasoning fidelity and valid inter-step causal dependencies. However, prior work suggests that CoT traces can be unfaithful, serving as post-hoc rationalizations rather than reflections of the model's internal decision-making process. This paper challenges the fundamental assumption that current probabilistic metrics truly capture the logical structure and causal dependencies necessary for valid reasoning, questioning whether they merely measure surface-level fluency or in-distribution priors.

Methodology

To investigate whether probabilistic confidence metrics rely on inter-step causality or local fluency, the authors introduce a framework of causality perturbations. These perturbations systematically disrupt the logical flow and dependencies between reasoning steps while preserving local sentence fluency and semantic plausibility.

The study evaluates three distinct classes of disruptions across five reasoning benchmarks (MATH-500, GPQA-Diamond, GSM8K, LogiQA, AR-LSAT) and three LLM families (Qwen, Llama, Phi):

1. Attention-Level Disruptions:

   • Masking: A hard attention mask is applied between reasoning steps during metric calculation. This forces the model to evaluate each step independently ($p(s_k | x, s_{<k})$ becomes $p(s_k | x, s_{<k}^{local})$), breaking the autoregressive causal chain where step $k$ depends on step $k-1$.
   • Query Masking: An extreme case where conditioning on the input query is also removed, evaluating steps solely on their internal token history.

2. Parameter-Level Disruptions:

   • The evaluation is performed using a significantly smaller model (e.g., Qwen2.5-0.5B) compared to the generator (e.g., Qwen3-8B). Smaller models retain strong priors for common answer formats and local fluency but lack the capacity to model complex, long-range causal dependencies.

3. Data-Level Disruptions:

   • Paraphrasing: Each reasoning step is independently rewritten to preserve semantic meaning but alter syntactic and lexical signals linking steps.
   • Shuffling: The order of reasoning steps is randomized, destroying the sequential causal progression while keeping individual steps fluent.
   • Truncation: The latter portions of the reasoning trace are removed, testing reliance on complete multi-step chains versus initial prefixes.

The authors measure selection accuracy (the fraction of questions where the selected output is correct) under these disrupted conditions and compare them against the original, unaltered metrics.

Key Contributions

1. Empirical Evidence of Insensitivity: The paper provides strong evidence that state-of-the-art probabilistic confidence metrics are largely insensitive to disruptions in inter-step causal structure. Across diverse models and benchmarks, selection accuracy degrades only marginally (often <1%) or even improves when causal dependencies are severed.
2. Fluency vs. Reasoning: The findings suggest that current metrics primarily capture local fluency and answer format priors rather than the validity of the reasoning process. The metrics remain robust even when the logical coherence of the trace is destroyed.
3. Contrastive Causality Metric: To address this gap, the authors propose a new metric, $R_{causal}$, defined as:
   $$R_{causal}(y) = R(y) - \alpha \cdot \hat{R}(y)$$
   Where $R(y)$ is the original metric and $\hat{R}(y)$ is the metric computed under attention-level masking (representing local fluency). By subtracting the fluency component, this metric explicitly isolates signals attributable to inter-step causal dependencies.

Results

• Attention-Level: Masking cross-step attention resulted in minimal accuracy loss. For Qwen, masking even yielded a slight improvement (0.75%). Even for Phi, which showed larger drops (~6-7%), performance remained well above random baselines.
• Parameter-Level: Using a smaller evaluator model (0.5B vs 8B) resulted in negligible changes in selection accuracy (e.g., Qwen changed by only 0.7% on average), indicating that fine-grained reasoning capacity is not required for these metrics to function effectively.
• Data-Level:
  • Shuffling: Randomizing the order of steps caused no systematic degradation; in some cases, accuracy improved.
  • Paraphrasing: Lexical variations preserving sentence meaning led to negligible performance drops.
  • Truncation: Removing up to 30k tokens from the end of traces (leaving only short prefixes) did not substantially degrade accuracy, suggesting that the full reasoning chain is not necessary for effective ranking.

Significance and Claims

The paper claims that the reliance on probabilistic confidence metrics for Best-of-N selection may overestimate the reliability of these metrics as proxies for reasoning correctness. The pronounced lack of sensitivity to causal disruptions implies that these metrics are not faithfully capturing multi-step reasoning coherence.

Consequently, the authors argue that the field must move beyond simple probability-based selection. The proposed contrastive causality metric demonstrates that explicitly isolating causal dependencies yields more faithful output selection. The work highlights a critical misalignment between current evaluation practices and the actual logical structure of reasoning, urging a re-evaluation of how "reasoning quality" is measured in generative AI.