🤖 machine learning

Test-Time Scaling Makes Overtraining Compute-Optimal

This paper introduces Train-to-Test ( $T^2$ ) scaling laws that jointly optimize model size, training tokens, and inference samples under fixed budgets, revealing that accounting for test-time costs shifts optimal pretraining strategies into an overtrained regime that significantly outperforms standard scaling approaches.

Original authors: Nicholas Roberts, Sungjun Cho, Zhiqi Gao, Tzu-Heng Huang, Albert Wu, Gabriel Orlanski, Avi Trost, Kelly Buchanan, Aws Albarghouthi, Frederic Sala

Published 2026-04-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nicholas Roberts, Sungjun Cho, Zhiqi Gao, Tzu-Heng Huang, Albert Wu, Gabriel Orlanski, Avi Trost, Kelly Buchanan, Aws Albarghouthi, Frederic Sala

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a bakery. For years, the "experts" (the old scaling laws) told you exactly how to bake your cakes to get the best result for your money. Their rule was simple: "If you have a certain amount of flour and sugar (compute budget), buy a medium-sized oven and bake exactly 20 cakes. Don't bake more, don't bake fewer." This was the "Chinchilla" rule.

But there was a problem. The experts forgot to ask: "How are people actually eating these cakes?"

In the real world, if you are baking a cake for a simple request like "What's the weather?", one cake is fine. But if you are baking for a complex request like "Solve this difficult math problem," you might want to bake 100 cakes, taste them all, and pick the single best one.

The old rules didn't account for this. They assumed you only ever baked one cake per order.

The New Discovery: "Train-to-Test" (T2) Scaling

This paper introduces a new way of thinking called Train-to-Test (T2) Scaling. It connects the baking (training) with the eating (testing/inference).

Here is the core idea, broken down with analogies:

1. The Old Way vs. The New Way

The Old Way (Chinchilla): "Buy a huge, expensive oven (large model) and bake just one perfect cake. It's expensive to run the oven, but you only do it once."
The New Way (T2): "Buy a smaller, cheaper oven (smaller model). Because the oven is cheap, you can afford to bake 1,000 cakes for the same price. Then, you taste all 1,000 and serve the absolute best one to the customer."

2. The "Overtraining" Secret

The paper's biggest shocker is about Overtraining.

Old Rule: Stop baking when you've used 20 cups of flour per cake. If you use more, you are "wasting" money.
New Rule: If you know you are going to bake 1,000 cakes for every order, you should actually use way more flour on a smaller cake. You should "overtrain" the small model.

Why?
Think of a small, overtrained model like a highly specialized genius.

A large model is like a generalist who knows a little bit about everything. They are expensive to hire.
A small, overtrained model is like a specialist who has practiced one specific skill until they are perfect. They are cheap to hire.

If you hire the specialist and ask them to try solving a problem 100 times, they will eventually get it right. If you hire the expensive generalist and ask them to try once, they might get it wrong.

The paper proves that hiring a cheap specialist and asking them to try 100 times is often better and cheaper than hiring one expensive generalist.

3. The "Pass@k" Metaphor

The paper uses a metric called Pass@k.

Imagine you are taking a multiple-choice test.
k=1: You get one guess. If you get it wrong, you fail.
k=100: You get 100 guesses. Even if you are a bit shaky, the odds are almost 100% that one of those 100 guesses will be right.

The paper shows that if you plan to use k=100 (try 100 times), you should train your model differently. You should make the model smaller but train it much longer (more "flour") so that even a single guess is decent, and 100 guesses are perfect.

The Three Big Takeaways

Stop following the "20-to-1" rule blindly.
If you know you will be running your AI many times for each question (like in complex reasoning tasks), the old rule of "20 training tokens per parameter" is wrong. You should actually train smaller models for much longer.
Small and Overtrained is the new "Goldilocks."
The paper found that the best models are often smaller than we thought, but they are overtrained (trained on way more data than the old rules suggested). This makes them cheap to run, and because you run them many times, the quality skyrockets.
It works even after "Fine-Tuning."
Sometimes, after training a model, you tweak it for specific jobs (like writing code or answering medical questions). The paper checked if their new rule still worked after this tweaking. Yes, it did. The "small and overtrained" strategy remains the winner even after the model is specialized.

Summary in One Sentence

Don't just build one giant, expensive brain; instead, build a smaller, cheaper brain, train it until it's a master of its craft, and then ask it to try the same problem a hundred times to guarantee the perfect answer.

This approach saves money and gets better results, changing how we should build the AI of the future.

1. Problem Statement

Current Large Language Model (LLM) development is governed by two largely isolated paradigms:

Pretraining Scaling Laws (e.g., Chinchilla): These optimize model size ( $N$ ) and training tokens ( $D$ ) to minimize training loss under a fixed compute budget. They assume a single-pass inference scenario ( $k=1$ ).
Test-Time Scaling: Recent work shows that allocating more compute to inference via repeated sampling (generating $k$ samples and selecting the best) can significantly boost performance, often allowing smaller models to outperform larger ones.

The Gap: Existing scaling laws do not jointly optimize pretraining decisions ( $N, D$ ) with test-time inference strategies ( $k$ ).

The Trade-off: Smaller models are cheaper per inference sample but have lower per-sample accuracy. Larger models are expensive per sample but more accurate. The benefit of repeated sampling is a highly nonlinear function of per-sample quality.
The Question: If a practitioner knows their total compute budget includes both training and a specific test-time scaling budget (repeated sampling), should the optimal pretraining strategy (model size and token count) change? The paper argues that current "Chinchilla-optimal" ratios (approx. 20 tokens/parameter) are suboptimal when inference costs are accounted for.

2. Methodology: Train-to-Test (T2) Scaling Laws

The authors propose Train-to-Test (T2) scaling laws, which jointly optimize model size ( $N$ ), dataset size ( $D$ ), and the number of inference samples ( $k$ ) under a fixed end-to-end compute budget ( $C_{total} = C_{train} + C_{inf}$ ).

Cost Model:

Training Cost: $C_{train} \approx 6ND$ (standard Chinchilla assumption).
Inference Cost: $C_{inf} \approx 2Nk$ (forward pass cost $\times$ number of samples).
Constraint: For a fixed inference budget, smaller models are allocated more samples ( $k = C_{inf} / 2N$ ).

The authors propose two complementary modeling approaches to solve the optimization problem:

Approach 1: Parametric Model of Task Loss (NLL-based)

Concept: Extends the Chinchilla loss law to include the number of samples $k$ .
Formulation: The authors model the negative log of the pass@k probability as a power-law function:
$\hat{L}(N, D, k) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta} + \frac{G}{k^\gamma}$
Logic: This assumes that the reduction in negative log pass@k due to repeated sampling follows a power law (supported by prior work assuming a Beta distribution of task difficulties).
Optimization: Minimizes $\hat{L}$ subject to $6ND \le C_{train}$ and $2Nk \le C_{inf}$ .

Approach 2: Parametric Model of Task Accuracy (Pass@k-based)

Concept: Directly models the expected pass@k accuracy rather than loss.
Formulation:
1. Maps the Chinchilla loss $\hat{L}(N, D)$ to a single-pass accuracy $p$ using a fitted sigmoid function: $p = \sigma_\theta(\hat{L}(N, D))$ .
2. Models the distribution of per-question accuracies across the dataset as a Beta distribution ($Beta(a, b)$), where parameters $a, b$ depend on $N$ and $D$ .
3. Derives the expected pass@k by integrating over this Beta distribution:
  $\widehat{Acc}(N, D, k) = 1 - \frac{B(a, b+k)}{B(a, b)}$
  where $B$ is the Beta function.
Optimization: Maximizes $\widehat{Acc}$ subject to the same compute constraints.

3. Key Contributions

Unified Framework: Formalized the joint optimization of pretraining ( $N, D$ ) and test-time scaling ( $k$ ) under a single compute budget.
Dual Modeling Approaches: Introduced two distinct methods (Loss-based and Accuracy-based) that, despite modeling different quantities, yield consistent recommendations.
Validation on Overtrained Regime: Built a testbed of over 100 models (extending existing Chinchilla suites into the overtrained regime) to empirically validate that the predicted optima are real and not just theoretical artifacts.
Post-Training Robustness: Demonstrated that the T2 scaling benefits persist after Supervised Fine-Tuning (SFT) and standard Fine-Tuning (FT), even though overtrained models are known to be harder to fine-tune.

4. Experimental Results

The authors evaluated T2 scaling across eight downstream tasks (4 real-world benchmarks: LAMBADA, ARC-Easy, SciQ, OpenBookQA; and 4 synthetic tasks: knowledge recall, arithmetic, causal reasoning, spatial reasoning).

RQ1: Should pretraining change?
- Yes. When accounting for a fixed test-time scaling budget, the optimal strategy shifts radically toward smaller, heavily overtrained models.
- Instead of the Chinchilla rule of ~20 tokens/parameter, T2 recommends ratios often exceeding 100–1000 tokens/parameter (depending on the budget).
- The optimal frontier shifts to smaller models ( $N$ ) that are trained on significantly more tokens ( $D$ ) and evaluated with many more samples ( $k$ ).
RQ2: Does T2 extrapolate to overtrained checkpoints?
- The authors trained 21 new "overtrained" checkpoints from scratch to validate the predictions.
- Result: The T2 models consistently outperformed the Chinchilla-optimal checkpoints.
- Accuracy: Approach 1 (NLL) had a relative error of 2.8% in predicting performance, while Approach 2 had 8.4%.
- Performance: Small overtrained models (e.g., 37M–149M parameters) with repeated sampling significantly outperformed large Chinchilla-optimal models (e.g., 901M parameters) on the same tasks.
RQ3: Does T2 survive post-training?
- The authors applied FT and SFT to the best overtrained and Chinchilla-optimal models.
- Result: The performance gap persisted. The overtrained models remained superior even after fine-tuning, though the magnitude of the "overtraining" benefit was slightly subdued compared to base models (consistent with findings that overtrained models are harder to fine-tune).

5. Significance and Implications

Redefining "Compute-Optimal": The paper challenges the prevailing Chinchilla scaling law, arguing that it is only optimal for single-pass inference. For modern deployments utilizing test-time compute (repeated sampling, verification, or reasoning), the optimal strategy is to train smaller models for much longer.
Practical Recommendation: Practitioners with a fixed inference budget should not train massive models for a single pass. Instead, they should train smaller models with high token-to-parameter ratios and rely on repeated sampling at inference time to achieve better accuracy per dollar.
Bridging Theory and Practice: This work connects the theoretical scaling laws of pretraining with the practical realities of deployment strategies (like those used in frontier models for reasoning), providing a blueprint for efficient model design in the era of test-time scaling.

In summary, the paper proves that overtraining is not a mistake but a necessity when test-time compute is part of the optimization equation, offering a new "Train-to-Test" scaling law that maximizes performance under realistic deployment constraints.