Technical Summary: "More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It"

1. Problem Statement

The paper addresses a critical paradox in test-time scaling for Large Language Models (LLMs): Power Sampling can increase the total probability mass assigned to correct reasoning trajectories while simultaneously degrading the final aggregated answer quality.

Power Sampling sharpens a model's distribution over complete generation trajectories by exponentiating the probability of each trajectory ($p(\tau)^\alpha$). While this method is verifier-free and aims to favor likely correct paths, the authors observe that it often leads to significant accuracy drops (up to 18.5 percentage points) when used with downstream aggregation methods like Self-Consistency (SC).

The core issue is a misalignment between trajectory-level sharpening and multi-trajectory decision rules. The paper identifies two specific mismatches causing this failure:

1. Coverage Mismatch: Power Sampling rewards individual trajectories based on their raw likelihood. In aggregation methods (like SC), the final answer depends on the collective support of multiple trajectories mapping to the same answer. Power Sampling can amplify a single, high-likelihood incorrect trajectory (e.g., $W_1$) while suppressing a set of moderate-likelihood correct trajectories (e.g., $C_1, C_2, C_3$). Consequently, the aggregated decision flips to the wrong answer, even though the total mass of correct trajectories may have increased. High pass@k (the probability that at least one correct trajectory exists) remains high, but the distribution fails to preserve the broad support necessary for aggregation.
2. Dose Mismatch: A fixed exponent $\alpha$ applies the same algebraic transformation across all problems, but the "dose" of distributional change varies drastically. Because the geometry of likelihoods differs across prompts, the same $\alpha$ can induce a mild adjustment on one problem but a near-collapse of the distribution on another. This makes the sharpening effect unpredictable and difficult to control globally.

2. Methodology

The authors propose Relative-Rank SoftSat, a repair mechanism that decouples the control of sharpening strength (dose) from the preservation of diversity (coverage).

2.1 Theoretical Foundation: Weighted Consensus

To avoid the computational cost of running Power Sampling (e.g., Power-SMC) $N$ times for $N$ samples, the authors establish a theoretical link between ideal target sampling and importance-weighted aggregation.

• They show that the answer mass of an ideal Power target can be estimated from a single pool of $N$ trajectories sampled from the Base model ($p$).
• By using importance weights $r_\alpha(\tau) = p(\tau)^{\alpha-1}$, the weighted consensus of the Base pool asymptotically converges to the decision of direct target sampling under standard conditions. This allows the method to reuse the same generation budget as standard Self-Consistency.

2.2 Relative-Rank SoftSat Design

The proposed sampler replaces the raw log-likelihood $\ell(\tau)$ with a within-problem relative rank $u_p(\tau)$, defined as the percentile of a trajectory's likelihood among all trajectories for that specific prompt.

The reweighting multiplier is defined as:
$$r_{\beta,m}(\tau) = \exp[\beta \cdot \text{SoftSat}_m(u_p(\tau))]$$

Where $\text{SoftSat}_m(u)$ is a saturating function:
$$\text{SoftSat}_m(u) = 1 - \left(1 - \min\left\{\frac{u}{m}, 1\right\}\right)^2$$

Key Mechanisms:

• Addressing Dose Mismatch: By using the relative rank $u \in [0, 1]$ instead of raw likelihood, the method normalizes the "dose" of sharpening across different problems. A fixed hyperparameter $\beta$ now applies a comparable transformation regardless of the prompt's specific likelihood geometry.
• Addressing Coverage Mismatch: The SoftSat function is designed to saturate gains for top-ranked trajectories (preventing a single dominant path from absorbing disproportionate mass) while boosting moderately ranked trajectories. This preserves the broad support required for downstream aggregation.

2.3 Implementation

The algorithm operates as follows:

1. Sample $N$ trajectories from the Base model.
2. Compute empirical relative ranks within the pool.
3. Apply the SoftSat gain and clip weights to ensure stability.
4. Perform a weighted consensus readout (e.g., Weighted Self-Consistency or ModeX) using these weights.

3. Key Contributions

1. Diagnosis of Failure: The paper identifies and characterizes coverage mismatch and dose mismatch as the root causes of why Power Sampling degrades multi-sample inference, despite increasing correct-trajectory mass.
2. Relative-Rank SoftSat: The introduction of a new sampling target that uses within-prompt rank to control sharpening strength and a saturating gain function to preserve trajectory diversity.
3. Efficient Realization: A method to realize the ideal target sampling via importance-weighted aggregation over a shared Base candidate pool, maintaining the computational efficiency of parallel sampling without requiring repeated Power-SMC runs.

4. Experimental Results

The authors evaluated the method across three benchmarks (BigCodeBench, LiveAoPSBench, PHYSICS) and three models (Nemotron-4B, Qwen3.5-9B, Ministral-8B) using a fixed budget of 8 trajectories per problem.

• Power Sampling Failure: Fixed-exponent Power Sampling ($\alpha=4$) degraded performance in 7 out of 9 settings. Notably, it caused an 18.5 percentage point drop on LiveAoPSBench with Nemotron-4B.
• SoftSat Performance:
  • SoftSat eliminated the large regressions caused by Power Sampling. On the worst Power failure cases (Nemotron on LiveAoPS and PHYSICS), SoftSat reduced the performance gap relative to Uniform Self-Consistency from ~18.5 points to ~1.0 point and ~7.0 points to ~0.25 points, respectively.
  • SoftSat improved performance in 5 out of 9 settings compared to Uniform Self-Consistency, with a maximum regression of only 1.004 points.
  • Unlike Power Sampling, SoftSat demonstrated stability across different response spaces (code vs. math vs. physics) and models.
• Budget Dependence: Experiments showed that the optimal sharpening strength ($\beta$) increases with the number of available trajectories ($N$), suggesting that larger candidate pools can support stronger reweighting rules.

5. Significance and Claims

The paper claims that trajectory-level sharpening is not reliably aligned with multi-trajectory decision rules. The significance of this work lies in:

• Correcting a Misconception: It challenges the assumption that simply increasing the probability mass of correct trajectories (via Power Sampling) automatically improves downstream reasoning performance.
• Preserving Diversity: It demonstrates that for aggregation-based methods, preserving the "collective support" of correct answers is more critical than maximizing the probability of the single most likely trajectory.
• Practical Repair: It offers a computationally efficient, verifier-free alternative (SoftSat) that calibrates sharpening across problems, preventing the "winner-take-all" collapse that harms Self-Consistency and similar aggregation strategies.

The authors conclude that while Power Sampling is a powerful tool for distribution shaping, its application to multi-sample inference requires careful calibration of both the shape (to preserve coverage) and the strength (to ensure consistent dose) of the sharpening.