💬 NLP

T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Models

This paper introduces Tool-integrated Verification (T1), a framework that enhances test-time compute scaling for small language models by offloading memorization-heavy verification tasks to external tools, enabling a 1B model to outperform a significantly larger 8B model on the MATH benchmark.

Original authors: Minki Kang, Jongwon Jeong, Jaewoong Cho

Published 2026-06-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Minki Kang, Jongwon Jeong, Jaewoong Cho

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Small Brain" vs. The "Big Math Test"

Imagine you have a very smart but small student (a Small Language Model, or sLM). This student is great at writing stories and understanding general ideas, but they struggle with heavy lifting, like complex math or remembering obscure facts.

Recently, researchers discovered a trick called Test-Time Compute Scaling. Instead of making the student bigger (which is expensive and slow), you let them take a test multiple times, generate many different answers, and then pick the best one.

The Catch: To pick the best answer, you need a Judge.

The Old Way: You hire a giant, super-smart professor (a Large Language Model) to grade the small student's work. This works well, but it defeats the purpose of using a small, cheap student in the first place.
The New Problem: What if the small student tries to grade their own work? The paper shows that when the math gets hard (like adding three-digit numbers), the small student's "brain" gets overloaded. They can't remember all the calculation facts, so they make mistakes in grading, just like they did in solving the problem.

The Solution: T1 (The "Calculator + Teacher" Team)

The authors propose a new system called T1 (Tool-Integrated Verification). Think of this as giving the small student a calculator and a fact-checker before they try to grade their own work.

Here is how T1 works in two simple steps:

Step 1: The "Calculator Filter" (Tool-Based Verifier)

Before the small student even looks at the answers, they use an external tool (like a code interpreter or a search engine) to check the hard stuff.

The Analogy: Imagine the student is checking a math test. Instead of trying to do $237 + 321$ in their head (where they might mess up), they are forced to write a tiny piece of code to let a computer do the math.
The Result: The computer instantly says, "This answer is wrong because the math is bad," and throws that answer in the trash. The small student doesn't have to strain their brain to remember the numbers; the tool does it for them.

Step 2: The "Teacher's Review" (Reward Model)

Now, the small student only has to grade the answers that passed the calculator filter. They use their own "teacher brain" (a Reward Model) to check if the logic makes sense, if the story flows, and if the reasoning is sound.

The Analogy: Since the calculator already removed the answers with bad math, the student can focus entirely on the logic. "Okay, the math is right, but did they answer the specific question asked?"

Why This Works (The "Memory" Magic)

The paper proves a fascinating theory: Small models fail at verification because they have to memorize too many facts.

Without Tools: To verify a math problem, the small model has to "remember" the answer to $237 + 321$ inside its own head. As the problems get harder, its memory runs out.
With Tools: The model doesn't need to memorize the answer. It just needs to know how to ask the calculator. This frees up its brain to focus on the harder task of logical reasoning.

The Results: Small Beating Big

The researchers tested this on difficult math benchmarks (like MATH and GSM8K).

The Shocking Result: A tiny 1-billion parameter model (the small student) using T1 performed better than a much larger 8-billion parameter model (the big student) that didn't use tools.
The Takeaway: By offloading the "boring memory work" to tools, the small model becomes a much better judge. It can now verify its own work so well that it beats models that are eight times larger.

Summary

Think of T1 as giving a small, efficient worker a specialized toolkit. Instead of trying to be a "super-human" who remembers everything and does everything, the worker uses a calculator to handle the heavy lifting (math/facts) and then uses their own brain to handle the judgment (logic). This allows small, cheap models to perform at a level previously thought impossible without massive, expensive computers.

Technical Summary: Tool-Integrated Verification for Test-Time Compute Scaling in Small Language Models

Problem Statement

Recent advancements in test-time compute scaling have demonstrated that allocating additional computation during inference can significantly enhance the reasoning capabilities of Small Language Models (sLMs). However, existing approaches to scaling rely heavily on large language models (LLMs) acting as verifiers (e.g., Process Reward Models or Critic models) to select the best solution from multiple candidates. This reliance on large verifiers undermines the efficiency and cost benefits of using sLMs.

A critical bottleneck identified in this work is the inability of sLMs to reliably verify their own generated outputs, particularly in tasks requiring memorization of arithmetic facts or factual knowledge. While larger models can verify solutions using chain-of-thought reasoning alone, sLMs struggle with even simple calculations as complexity increases. The authors hypothesize that this failure stems from the limited capacity of sLMs to memorize the vast number of calculation facts required for verification, rather than a lack of logical reasoning capability.

Methodology: Tool-Integrated Verification (T1)

To address the memorization bottleneck, the authors propose Tool-Integrated Verification (T1), a two-stage framework designed to offload memorization-heavy steps to external tools while retaining the sLM for final verification.

The Two-Stage Framework

Tool-Based Filtering Stage (ToolV):
- The sLM generates executable code (e.g., Python) or retrieval queries based on the candidate solution.
- An external tool (such as a code interpreter or knowledge retriever) executes the code or retrieves facts to verify numerical accuracy and factual consistency.
- This stage acts as a binary filter ( $f(x, y; T, \theta) \in \{0, 1\}$ ), discarding solutions with calculation errors or incorrect facts.
Reward Model-Based Scoring Stage:
- The remaining filtered solutions are scored by a reward model (either a Process Reward Model [PRM] or a Generative Reward Model [GenRM]).
- The final output is selected as the candidate with the highest score from the reward model.

Knowledge Distillation

To enhance the performance of both stages, the authors employ knowledge distillation from a larger teacher model (e.g., GPT-4o-mini).

ToolV Distillation: The sLM is fine-tuned to generate tool-calling queries (code) and extract verification results based on the teacher's outputs.
RM Distillation: The sLM is fine-tuned to generate verification rationales and scores, mimicking the teacher's evaluation.
Multi-LoRA: Separate Low-Rank Adaptation (LoRA) adapters are used for the tool-based and reward-based tasks to efficiently manage distinct objectives during fine-tuning.

Theoretical Analysis

The paper provides theoretical guarantees supporting the T1 approach:

Memorization Reduction: Using information-theoretic analysis, the authors prove that without tools, an optimal verifier must memorize a dataset of size proportional to $O(M^3)$ (where $M$ is the range of numbers) to achieve near-zero error. By utilizing an external tool to perform the calculation, the mutual information required to be stored in the model parameters drops to zero, as the tool handles the verification logic.
Scaling Performance: The authors model the verifier as an imperfect filter with noise. They demonstrate that using a tool-based filter increases the probability of correctly identifying the ground truth label in a Best-of-N selection process. Theoretically, this improves the monotonicity of test-time scaling performance, ensuring that increasing the number of generated solutions ( $N$ ) yields better results when a reliable filter is present.

Experimental Results

Experiments were conducted on mathematical reasoning benchmarks (MATH500, GSM8K) and knowledge-intensive tasks (MMLU-Pro subset).

Performance Gains: T1 enables sLMs to outperform significantly larger models. Specifically, a Llama-3.2 1B model using T1 under test-time scaling outperforms the Llama-3.1 8B model on the MATH500 benchmark.
Integration with Verifiers: T1 improves the accuracy of both PRMs and GenRMs. When combined with a distilled PRM, T1 allows a 1B model to match or exceed the performance of an 8B model with fewer generated solutions.
Filtering Efficacy: Analysis shows that ToolV effectively filters out incorrect solutions caused by calculation errors, which generative verifiers alone often miss.
Compute Efficiency: Even when accounting for the token overhead of generating code (approximately a 1.14x increase in compute budget), T1 provides superior performance compared to simply scaling up the verifier size or using larger models without tools.
Data Efficiency: The tool-based verifier maintains competitive performance even when distilled with only 10% of the training data, highlighting its efficiency.

Significance and Claims

The paper claims that tool integration is not merely beneficial but necessary for enabling successful verification in sLMs under test-time scaling. The key contributions are:

Identification of Bottleneck: Systematically identifying memorization-heavy steps as the primary failure point for sLM verification.
Framework Proposal: Introducing T1, a seamless two-stage framework that integrates external tools with sLMs for both generative verifiers and PRMs.
Theoretical Justification: Proving that offloading to tools reduces the memorization burden and theoretically guarantees improved scaling performance.
Empirical Validation: Demonstrating that small models (e.g., 1B parameters) can achieve reasoning capabilities comparable to much larger models (e.g., 8B+) by leveraging tools, thereby preserving the deployment efficiency of sLMs.

The authors conclude that T1 offers a practical pathway to enhance sLM performance in complex reasoning tasks without the prohibitive cost of scaling model parameters, provided that the verification burden is shifted to external tools.