💬 NLP

Model Capability Dominates: Inference-Time Optimization Lessons from AIMO 3

In the AIMO 3 competition, experiments demonstrate that model capability overwhelmingly dominates inference-time optimization strategies, as attempts to improve performance through diverse prompting or error decorrelation were consistently outperformed by simply using a more capable model.

Original authors: Natapong Nitarach

Published 2026-03-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Natapong Nitarach

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Super-Genius" vs. The "Group Strategy"

Imagine you are trying to solve the world's hardest math puzzles (like the International Math Olympiad). You have a team of AI "students" (Large Language Models) to help you.

For a long time, the winning strategy was The Group Vote. The idea was simple: "If one student gets it wrong, maybe another will get it right. So, let's ask the same student 8 times, give them slightly different instructions, and take the answer that most of them agree on."

This paper is a report from a math competition (AIMO 3) where the author tried to prove that changing the instructions (the "Group Strategy") is useless. Instead, the only thing that matters is how smart the student actually is (Model Capability).

The Experiment: The "Diverse Prompt Mixer"

The author, Natapong, set up a high-stakes race on a single, powerful computer chip (an H100 GPU) with a strict 5-hour time limit.

The Hypothesis:
If you ask a smart student to solve a problem using 8 different methods (e.g., "Start with small numbers," "Work backward," "Write code first"), their mistakes should be different. If their mistakes are different, the "majority vote" should be much smarter than if they all used the same method.

The Setup:

The Student: A massive AI model called gpt-oss-120b.
The Race: 50 incredibly hard math problems.
The Trick: The author tried mixing and matching 4 different "personality" prompts for the 8 attempts per problem.

The Results: The "Group Strategy" Failed

The author expected the "Diverse Prompt Mixer" to win. Instead, it lost. Here is why, using three simple analogies:

1. The "Randomness" Analogy (Temperature)

The author thought they needed to give the students different instructions to get different answers. But they realized the students were already acting differently just by being asked to think fast and loosely (high "temperature").

Analogy: Imagine asking a jazz musician to play a song. If you tell them, "Play it fast and loose," they will naturally play it differently every time. You don't need to give them a new sheet of music (a new prompt) to get variety; the "looseness" is enough.
Result: Adding different prompts didn't add new variety; it just confused the student and made them slower and less accurate.

2. The "Bad Luck" Analogy (Correlation)

The theory of the "Group Vote" relies on the idea that if one student makes a mistake, the others won't make the same mistake.

Analogy: Imagine a group of 8 people trying to guess the weight of a cow. If they all look at the cow and guess "500 lbs" because they all think it's a calf, their votes are correlated (they are wrong together).
The Surprise: The author found that even when the AI made a mistake, it didn't make the same mistake twice. The errors were scattered randomly. The "Group Vote" was already working perfectly well without the extra prompts. Trying to force diversity was like trying to shake a dice that is already rolling perfectly.

3. The "Weaker Student" Analogy (Accuracy Loss)

This is the most important finding. When the author switched the AI to "Work Backwards" or "Start with Small Cases," the AI actually got worse at solving the problem.

Analogy: Imagine you have a Grandmaster Chess player. You tell them, "Okay, for this next game, try to play like a beginner who only moves pawns." They will still be a Grandmaster, but they will play worse than if they just played normally.
Result: The "diverse" prompts were like forcing the Grandmaster to play with one hand tied behind their back. The slight benefit of having different opinions was completely wiped out by the fact that the student was now less capable.

The Real Winner: The "Super-Genius"

The paper compares three different AI models:

The Big One (gpt-oss-120b): A massive, highly capable model.
The Medium One (Qwen): Smaller and less capable.
The Other Big One (Nemotron): Also large, but with different architecture.

The Finding:
The difference in score between the "Big One" and the "Medium One" was huge (about 17 points out of 50).
The difference between using "smart prompts" vs. "dumb prompts" on the same model was tiny (about 1 or 2 points).

Analogy: It's like comparing a Ferrari to a bicycle.
- Model Capability: Putting a Ferrari engine in the car.
- Prompt Engineering: Polishing the car's paint job or changing the driver's playlist.
- The Paper says: No matter how much you polish the bicycle or change the playlist, it will never beat the Ferrari. The engine (the model's intelligence) is 10 times more important than the paint job (the prompts).

The Final Lesson: "Lottery Tickets"

Since the "Group Strategy" (mixing prompts) didn't help, how do you win?
The author suggests treating your attempts like Lottery Tickets.

The Strategy: Don't try to be clever with your prompts. Just take your best, smartest model, give it the standard instructions, and run it as many times as you can within the time limit.
Why? Because the "best" model is so smart that sometimes, just by pure luck (randomness), it will stumble upon the right answer. If you run it 13 times, you have 13 chances to get that lucky break. Trying to "optimize" the prompts just lowers your average score and reduces your chances of a lucky win.

Summary in One Sentence

You cannot fix a weak AI with clever instructions, and you don't need clever instructions for a strong AI; you just need the strongest AI possible and to run it as many times as you can.

1. Problem Statement

The paper addresses the challenge of improving Large Language Model (LLM) performance on competition-level mathematics (specifically the AI Mathematical Olympiad, AIMO 3) using inference-time optimization without additional training.

Context: The standard approach for mathematical reasoning is Majority Voting (Self-Consistency), where $N$ inference attempts are generated, and the most frequent answer is selected. This relies on the Condorcet Jury Theorem, assuming errors are independent.
The Bottleneck: In practice, errors are highly correlated ( $\rho > 0$ ). A model using the same prompt makes the same systematic mistakes regardless of random seeds, drastically reducing the effective sample size ( $N_{eff}$ ).
The Hypothesis: The authors hypothesized that assigning structurally different reasoning strategies (e.g., "Work Backwards," "Small Cases First") to different voters would decorrelate errors, thereby increasing $N_{eff}$ and improving the final score. They termed this approach the Diverse Prompt Mixer.

2. Methodology

The study was conducted under strict hardware and time constraints: a single NVIDIA H100 80 GB GPU with a 5-hour wall-clock limit to solve 50 IMO-level problems.

System Architecture

Model: gpt-oss-120b (116.8B total parameters, 5.1B active via MoE), served locally via vLLM with FP8 quantization.
Pipeline:
1. Budget Allocation: Time is divided equally among remaining problems to guarantee completion.
2. Parallel Inference: $N=8$ concurrent threads per problem.
3. Tool Integration: Each attempt can generate and execute Python code (using sympy, numpy, mpmath) in a persistent Jupyter kernel.
4. Voting: Entropy-weighted majority voting (confident, low-entropy answers receive higher weight).
Experimental Design:
- Baseline: 8 attempts with a standard "Step-by-Step" prompt at Temperature $T=1.0$ .
- Diverse Prompt Mixer: Combinations of four distinct system prompts:
  1. Original: Standard CoT.
  2. Small Cases First: Enumerate $n=1,2,3...$ to find patterns.
  3. Work Backwards: List constraints and construct the solution.
  4. Classify Then Solve: Identify problem type and apply canonical techniques.
- Cross-Model Validation: Tested on Qwen3.5-35B-A3B and Nemotron-Super-120B to rule out model-specific artifacts.
- Ablation: 23+ experiments modifying temperature, prompt diversity, attempt count ( $N$ ), and reasoning strategies.

3. Key Contributions & Findings

A. The Failure of Diverse Prompt Mixer

Contrary to the hypothesis, introducing prompt diversity degraded performance.

Result: The "Equal Mix" (2 of each strategy) scored 38/50, while the Baseline (all Original prompts) scored 39.7/50.
Observation: Every individual diverse strategy (e.g., "Small Cases") performed worse than the Original prompt when used exclusively. The relationship was monotonic: more diversity = lower score.
Reason: While diverse prompts intended to reduce error correlation ( $\rho$ ), they simultaneously reduced per-attempt accuracy ( $p$ ). The loss in accuracy outweighed any gains from decorrelation.

B. High-Temperature Sampling is Sufficient

The authors found that at $T=1.0$ , the model already explores sufficiently diverse reasoning paths even with identical prompts.

Evidence: Pairwise error correlation ( $\rho$ ) was measured to be negative (mean $\hat{\rho} \approx -0.11$ to $-0.26$) across models.
Implication: Errors are already decorrelated (or even anti-correlated) due to stochastic sampling. Adding prompt diversity is redundant and, because the diverse prompts are less robust to the model's training distribution, harmful.

C. Model Capability Dominates

The most significant finding is that model capability ( $p$ ) dominates inference-time tricks by an order of magnitude.

The Gap: There was a 17-point score gap between the best model (gpt-oss-120b, score ~39.7) and the next best (Nemotron/Qwen, score ~23).
Comparison: All inference-time optimizations tested (prompt mixing, temperature tuning, code-first strategies) yielded changes of only $\pm 2$ points.
Conclusion: No amount of prompt engineering or voting strategy can bridge a fundamental capability gap. A stronger model with a simple baseline outperforms a weaker model with complex optimization.

D. The "Lottery" Nature of Submission

With a baseline mean of 39.7 and standard deviation of 1.7, the probability of hitting a top-tier score (e.g., $\ge 44$ ) in a single run is low (~0.6%).

Strategy: The optimal strategy for competitions is not to optimize the prompt, but to treat submissions as lottery tickets. Repeatedly submitting the unmodified baseline maximizes the probability of hitting a high-score outlier.

4. Significance and Implications

Refutation of "Diversity for Diversity's Sake": The paper challenges the intuition that diverse reasoning strategies are always beneficial. For models near their capability frontier, prompt robustness is low; changing the instruction format shifts the model out of its training distribution, causing accuracy drops that outweigh decorrelation benefits.
Shift in Optimization Paradigm: The field is moving from High- $N$ (brute-force voting with many attempts, as seen in AIMO-1) to High- $p$ (using more capable models with fewer attempts). Inference-time scaling has diminishing returns once the base system is reasonably configured.
Training vs. Inference: The paper suggests that to truly benefit from diverse prompts, models must be trained on Mixture of Formats (MOF) data to learn prompt robustness. This is a training-time intervention, not an inference-time one.
Practical Guidance: For hardware-constrained environments:
- Use the largest model that fits in memory.
- Keep temperature high ( $T \approx 1.0$ ) to leverage natural stochastic diversity.
- Avoid complex prompt mixing; stick to the strongest, most familiar prompt format.
- Allocate budget to multiple submission attempts (lottery tickets) rather than complex per-attempt optimization.

Summary Table of Results (gpt-oss-120b)

Configuration	Strategy	LB Score (Mean)	$\Delta$ from Baseline
Baseline	8x Original Prompt ( $T=1.0$ )	39.7	—
Conservative Mix	5 Orig + 1 Small + 1 Back + 1 Class	40.0	+0.3
Aggressive Mix	3 Orig + 2 Small + 2 Back + 1 Class	40.0	+0.3
Equal Mix	2 of each strategy	38.0	-1.7
8x Small Cases	All "Small Cases First"	37.0	-2.7
8x Code-First	All "Code-First" (Unstable)	37.7 (3-run mean)	-2.0
Nemotron (Cross-Model)	3 Attempts	23.0	-16.7

Final Takeaway: In the realm of high-difficulty mathematical reasoning, model capability is the primary driver of success. Inference-time optimizations like prompt diversity offer negligible or negative returns when the base model is already operating near its limits.