💬 NLP

Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B

This paper demonstrates that across various open-source models (1.5B to 7B parameters) and mathematics benchmarks, repeated sampling consistently outperforms or matches complex self-refinement and reflection methods when evaluated at equal token costs, revealing that the latter's apparent gains often stem from increased generation volume rather than superior reasoning strategies.

Original authors: Iliya Mirzaei

Published 2026-07-31

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Iliya Mirzaei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Great AI Brain-Boost Experiment

Imagine you have a very smart but slightly distracted student. You give them a hard math problem, and they try to solve it. Sometimes they get it right; sometimes they get it wrong. In the world of artificial intelligence, researchers have been trying to figure out how to make these "students" (called Large Language Models) think better without rebuilding them from scratch. The popular idea has been to tell the AI to "think harder." This means asking it to plan its steps, argue with itself, critique its own mistakes, or rewrite its answers before handing them in. It's like telling a student, "Don't just write the answer; write a whole essay about why you think that answer is right, check your work, and maybe try again."

The big question is: Does all that extra thinking actually help, or is it just wasting time? The key concept here is tokens. Think of tokens as the "words" or "units of effort" an AI spends. Every time the AI writes a word, it costs a token. If a method makes the AI write 1,000 words to solve a problem that could be solved in 100, it has spent ten times the effort. The scientific debate is whether that extra effort comes from the strategy (like self-criticism) or simply from the fact that the AI got to write more words. If you give a student ten times as much paper, they might get a better grade just because they had more space to try, not because they suddenly became smarter. This paper sets out to test exactly that: when you give different strategies the exact same amount of paper (tokens), which one actually wins?

The Paper: More Samples, Less Reflection

This paper, titled "Sample More, Reflect Less," is a massive experiment designed to settle the argument about whether AI needs to "reflect" on its own work to get smarter. The researchers took seven different popular methods that make AI "think harder"—including methods where the AI critiques itself, argues with copies of itself, or tries to pick the best answer from a list—and pitted them against a very simple, boring baseline: repeated sampling.

The baseline is like flipping a coin ten times and picking the side that comes up most often. In AI terms, it means asking the same question ten times and just taking the most common answer. The researchers made sure every method used the exact same number of tokens (words generated) so the playing field was perfectly level. They tested this on three different sizes of AI models (1.5 billion, 3 billion, and 7 billion parameters) using two math benchmarks (GSM8K and MATH-500) with 150 questions each.

The Big Finding:
The results were surprising. In almost every case, the fancy methods that made the AI "reflect," "critique," or "argue" lost to the simple method of just asking the question multiple times and counting the votes.

When the researchers looked closely at why the fancy methods lost, they found a clear pattern. The methods that asked the AI to judge its own work (like rewriting its answer or picking the "best" one from a list) consistently performed worse than the baseline. For example, on the smaller 1.5-billion-parameter model, a method called "Best-of-N" (where the AI picks the best of 8 answers) scored about 8 to 11 percentage points lower than just counting the most common answer among those same 8 tries. Even on the larger 7-billion model, the gap disappeared: the difference became statistically indistinguishable from zero, meaning the "judge" was no longer clearly worse than the "counter," and the penalty was no longer distinguishable from zero.

The paper explicitly rules out the idea that "thinking harder" through self-reflection is the magic ingredient. Instead, it suggests that spending tokens on another attempt is a better investment than spending tokens on self-correction—at every size the researchers could test. If you have enough tokens to write one long, critical essay about your answer, you are better off using those tokens to write two or three fresh, independent answers and just picking the most popular one.

The "Ghost" Method and the Size Matters Lesson

One of the most playful and revealing parts of the experiment involved a method called Reflexion. This method is supposed to work like this: the AI tries to solve a problem, then asks itself, "Did I get this right?" If it says "No," it tries again. If it says "Yes," it stops.

The researchers found a funny glitch in the smallest model (1.5B). On this tiny model, the AI never said "No." It judged every single one of its first answers as "correct," even when they were wrong. So, the Reflexion method never actually triggered a retry. It silently collapsed into a single, cheap attempt. Because it was so cheap (it didn't spend extra tokens), it looked like it was doing well compared to the expensive methods. But when the researchers forced the AI to retry three times regardless of what it thought, the performance dropped significantly, proving that the "reflection" mechanism itself was actually hurting the score.

As the models got bigger (from 1.5B to 7B), the gap between "counting votes" and "letting the AI judge" started to shrink. At 7 billion parameters, the AI was good enough at judging that it stopped being harmful; the difference was no longer distinguishable from zero. However, the paper notes that even at this size, the AI wasn't better at judging than counting; it just stopped being worse. The researchers suggest that for an AI to truly beat the simple "vote counting" method, it might need to be trained specifically to be a judge, not just asked to judge on the fly.

What This Means for the Future

The paper concludes with a strong message for anyone building these AI systems: Don't overcomplicate things. If you have a budget of tokens (words) to spend, the most efficient way to get a correct answer is often to generate many different attempts and let the majority rule, rather than asking the AI to critique and rewrite its own work.

The authors also warn that some methods might look good on paper but fail in practice because they "hide" their true cost. If a method decides for itself when to stop (like Reflexion), it might stop immediately and save money, making it look efficient, but it's not actually doing the work it was designed to do.

Ultimately, this study suggests that for math problems with clear right-or-wrong answers, the "magic" of self-reflection is a myth—at least for the models tested so far. The most powerful tool isn't a smarter strategy; it's simply trying more times. As the researchers put it, spending tokens to reconsider an answer is a worse use of them than spending the same tokens on another attempt. The AI doesn't need to be told to think harder; it just needs to be given more chances to try.

Problem Statement
Many methods claim to improve language model reasoning without retraining by instructing the model to plan, critique, reflect, debate, or select among multiple attempts. A critical flaw in evaluating these methods is that they invariably increase the number of tokens generated compared to a single chain of thought. Since generating more text inherently increases accuracy, a performance gain over a single chain of thought does not prove the method's mechanism is effective; it may simply reflect the increased compute budget.

Previous work (Wang et al., 2026) argued that a simple baseline—repeated sampling with majority voting (self-consistency)—often outperforms complex reasoning strategies when compute budgets are matched. However, that study relied on point estimates without confidence intervals, significance tests, or sufficient sample sizes to distinguish small effects from noise. Furthermore, it did not test small models (<7B parameters), where the ability to self-evaluate is likely weaker.

Methodology
The authors conduct a controlled, statistically rigorous experiment comparing seven reasoning methods against a cost-matched repeated sampling baseline.

Models: Qwen2.5-1.5B-Instruct, Qwen2.5-3B-Instruct, and Qwen2.5-7B-Instruct.
Benchmarks: GSM8K (150 questions) and MATH-500 (150 questions), selected for automatically checkable answers.
Cost Metric: Total generated tokens (completion tokens), including all intermediate steps, critiques, reflections, and debate turns. Input tokens are excluded to focus on the method's generative cost, though the authors note that including input tokens would further penalize methods that re-read context.
Baseline Construction: Instead of running separate self-consistency experiments for every token count, the authors generate a pool of $K=16$ independent chains per question. They then subsample this pool to construct an accuracy-vs-cost curve for self-consistency at any integer $N \le 16$ . This allows for precise interpolation of the baseline accuracy at the exact token cost of any tested method.
Statistical Rigor:
- Comparisons are paired question-by-question.
- Confidence intervals are derived via paired bootstrap (10,000 resamples).
- P-values are corrected for multiple testing using the Holm–Bonferroni procedure.
- The study explicitly measures the "activation rate" of adaptive methods (e.g., how often Reflexion actually triggers a retry).

Methods Evaluated

Chain-of-Thought (CoT): Single greedy pass.
Plan-and-Solve: Single greedy pass with a planning step.
Self-Refine: Iterative critique and rewrite (3 rounds).
Reflexion: Adaptive retry based on self-assessment (original version) and a forced version (fixed 3 rounds).
Best-of-N + Self-Verify: Generate 8 samples, then ask the model to pick the best.
Multi-Agent Debate: 3 agents debating for 2 rounds.
Sampling Baseline: Self-consistency (majority vote) at matched token cost.

Key Results

No Method Beats the Baseline: In none of the 36 comparisons (7 methods $\times$ 6 settings) did any method significantly outperform the repeated sampling baseline at an equal token cost.
Self-Inspection is Harmful: All methods requiring the model to assess or rewrite its own output (Self-Refine, Reflexion, Best-of-N with self-verification) performed significantly worse than the baseline in 10 out of 18 comparisons. The remaining comparisons were statistically indistinguishable from the baseline but never superior.
Counting vs. Choosing: A specific comparison using identical samples (8 chains) showed that simply counting the most frequent answer (majority voting) outperformed asking the model to choose the best answer.
- At 1.5B and 3B, the "counting" approach was 5 to 17 percentage points more accurate than "choosing."
- At 7B, the gap narrowed to ~2 points and was no longer statistically distinguishable from zero, suggesting the penalty for self-verification diminishes as model size increases but does not reverse within the tested range.
Reflexion's Silent Degeneration: The original Reflexion implementation, which relies on the model to decide when to retry, failed to trigger a single retry on the 1.5B model (it judged every answer correct). Consequently, it effectively became a single chain of thought, scoring well only because it was cheap. When forced to run, it performed significantly worse than the baseline.
Debate Convergence: Multi-agent debate converged to unanimous agreement in 48–91% of cases, suggesting the extra tokens bought agreement rather than accuracy.

Significance and Claims
The paper claims to establish that the "extra computation" hypothesis is insufficient to explain the success of complex reasoning methods. The primary finding is that spending tokens on self-evaluation or self-correction is a less efficient use of compute than spending the same tokens on additional independent attempts.

The authors make three specific, modest claims:

Mechanism vs. Budget: The gain observed in prior literature is often attributable to the increased token budget, not the reasoning mechanism. When budgets are strictly matched, simple sampling wins.
Self-Evaluation Limit: For models in the 1.5B to 7B range, asking a model to verify its own work is less effective than a simple tally. The "verifier" (the model itself) is not reliable enough to override the majority vote, even at 7B.
Evaluation Protocol: Adaptive methods can silently degenerate into cheaper, simpler methods (as seen with Reflexion on small models) if the adaptive trigger fails. Evaluations must report the activation rate of such triggers, not just final accuracy.

The paper concludes that for mathematical reasoning with checkable answers, the most efficient strategy for a fixed budget is repeated sampling with majority voting, and that the "missing comparison" (a cost-matched sampling baseline) should be standard in future method papers. The authors release all code, prompts, and raw generations to allow independent verification.

The Great AI Brain-Boost Experiment

The Paper: More Samples, Less Reflection

The "Ghost" Method and the Size Matters Lesson

What This Means for the Future

More like this