💬 NLP

Teaching Language Models to Think in Code

The paper introduces ThinC, a framework that shifts language model reasoning from interleaved natural language and code to a code-centric paradigm where code itself acts as the primary reasoner, achieving state-of-the-art performance on competition-level math benchmarks by distilling code trajectories and employing supervised fine-tuning followed by reinforcement learning.

Original authors: Hyeon Hwang, Jiwoo Lee, Jaewoo Kang

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hyeon Hwang, Jiwoo Lee, Jaewoo Kang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a very tricky math puzzle. You have two ways to do it:

The Old Way (Interleaved Reasoning): You talk to yourself in English, figuring out the steps. "Okay, first I need to multiply 500 by 0.12..." Then, you get a little unsure, so you ask a calculator (the code tool) to check your work. But here's the catch: if you made a mistake in your English thinking (like saying 500 times 0.12 is 50 instead of 60), you might accidentally type that wrong number into the calculator. The calculator just does what you tell it; it doesn't know you made a mistake. It calculates based on your wrong number, and you get a wrong answer.
The New Way (THINC): You take a tiny moment to plan your strategy in English ("I need to find the area of this shape"), and then you immediately hand the entire job over to a robot programmer. The robot writes code, runs it, sees the result, writes more code based on that result, and keeps going until the answer pops out. You don't do any math in your head or in English sentences; the robot does all the heavy lifting.

This paper introduces THINC (Thinking in Code), a new way to teach AI models to solve math problems by letting the code itself be the thinker, rather than just a tool the AI uses to check its work.

The Problem with the Old Way

The authors say current AI models that mix English thinking with code (called Tool-Integrated Reasoning) have three main flaws, which they call "structural limitations":

The "Post-Hoc" Verifier: The AI often writes out the whole solution in English first, then runs code just to say, "Yep, that's right." The code isn't actually doing the thinking; it's just a stamp of approval at the end.
The "Silent Error": If the AI makes a math mistake in its English thoughts (like calculating 12% of 500 as 50 instead of 60), it might copy that wrong number into the code. The code then calculates with the wrong number, and the AI never realizes it. The error is "silent" because the code just follows orders.
The "Double Work": The AI often writes a long English explanation of how to solve the problem, and then writes code that does exactly the same thing. It's like writing a recipe in English and then writing the exact same recipe in French, just to prove you know how to cook. It's redundant and confusing.

The THINC Solution

THINC changes the rules of the game. Instead of the AI talking about the math, the AI talks in the math (via code).

The Plan: The AI starts with one short English sentence to set the strategy (e.g., "I will loop through numbers to find the answer").
The Work: After that one sentence, everything happens in code blocks. The AI writes a piece of code, runs it, sees the output, and then writes the next piece of code based on that output.
The Result: The final answer comes directly from the computer's calculator (the interpreter), not from the AI guessing in English.

How They Taught the AI

The researchers didn't just tell the AI to do this; they taught it through a three-step process:

Distillation (The Teacher): They took a very smart, large AI model and asked it to solve math problems using this new "Code-First" style. They collected 12,200 examples of these perfect "Code-First" solutions.
Supervised Fine-Tuning (The Student): They taught two smaller AI models (one with 1.7 billion "brain cells" and one with 4 billion) to mimic these examples. This taught the models the habit of thinking in code.
Reinforcement Learning (The Coach): They let the models practice on thousands of math problems. If the model got the right answer, it got a "reward." If it failed, it learned to try a different code strategy. This made the models much smarter and more reliable.

The Results: Small Models, Big Wins

The paper tested these new models on very hard, competition-level math contests (like AIME and HMMT).

Beating the Giants: The small 4-billion-parameter THINC model scored 78.1% on average. This is better than the 1.7B model, better than other "Tool-Integrated" models, and even better than a massive 235-billion-parameter model that only thinks in English!
Reliability: In 99.2% of the cases, the final answer was directly pulled from the computer's code output. The AI didn't guess; it calculated.
Bouncing Back: If the code made a mistake and crashed (an error), the THINC model was surprisingly good at fixing it in the next code block without needing to "talk" its way out of it. Other models that mix English and code tend to fall apart when they hit a code error.

The Bottom Line

The paper claims that by forcing the AI to stop "talking" about the math and start "doing" the math in code, the models become more accurate, make fewer silly arithmetic mistakes, and solve hard problems more efficiently. It's like switching from a human trying to do long division in their head to a human who knows how to program a calculator to do it for them, step-by-step, without ever making a mental math error.

Technical Summary: Teaching Language Models to Think in Code (THINC)

Problem Statement

While Tool-Integrated Reasoning (TIR) has become a dominant paradigm for mathematical problem solving in Large Language Models (LLMs), the prevailing "interleaved" setup—where natural language (NL) reasoning alternates with code execution—suffers from three structural limitations:

Post-hoc Verification: Models often complete derivations in NL and use code merely to confirm the answer, rendering the interpreter a passive verifier rather than an active reasoner.
Unreliable NL Computation: Intermediate arithmetic or algebraic steps performed in NL are error-prone. When these erroneous values are copied into subsequent code blocks as hard-coded constants, the interpreter cannot detect the mistake, leading to silent error propagation.
Misallocated Roles: NL and code often perform overlapping tasks. NL frequently describes algorithms step-by-step that code subsequently re-implements, failing to leverage the distinct strengths of each modality (NL for high-level planning, code for precise execution).

Methodology

The authors propose THINC (Thinking in Code), a training framework designed to shift the primary reasoning burden from natural language to code execution.

1. The THINC Trajectory Format

Unlike standard TIR trajectories ( $q, t_1, c_1, o_1, t_2, c_2, o_2, \dots$ ), a THINC trajectory ( $q, t_1, c_1, o_1, c_2, o_2, \dots$ ) enforces a strict structural separation:

Single Planning Step ( $t_1$ ): The model generates one brief NL block at the start to define the high-level strategy or algebraic restructuring of the problem. Crucially, this block contains no step-by-step derivation or numerical computation.
Code-Centric Reasoning: All subsequent reasoning unfolds through code blocks ( $c_i$ ) connected only by their execution outputs ( $o_i$ ). There are no intermediate NL thought blocks between code executions.
Interpreter as Reasoner: The code blocks perform the derivations, self-corrections, and verifications. The final answer is derived directly from the interpreter's output.

2. Training Pipeline

The framework is realized through three stages:

Trajectory Distillation: A teacher model (Qwen3.5-27B) is prompted with few-shot examples to generate 12,200 code-centric trajectories. These are filtered to ensure correctness, successful execution, a minimum of three code blocks, and that the planning thought constitutes less than 50% of the total tokens.
Supervised Fine-Tuning (SFT): Student models (Qwen3-1.7B and Qwen3-4B-Thinking-2507) are fine-tuned on the distilled THINC-SFT dataset. This establishes the "code-centric behavior prior," teaching the model to adhere to the specific trajectory format.
Reinforcement Learning (RL): Using Group Relative Policy Optimization (GRPO) with verifiable rewards (exact match to ground truth), the models are further refined on the DAPO-Math-17k dataset. The RL process employs a three-stage curriculum with increasing context lengths (16K to 32K) and tool call limits to handle harder problems.

Key Contributions

THINC Framework: A novel training paradigm that treats code as the primary reasoner rather than a tool invoked by NL. It resolves the structural limitations of interleaved TIR by design.
THINC-SFT Dataset: A curated dataset of 12.2k code-centric trajectories demonstrating the desired reasoning format.
THINC Models: Two trained models, THINC-1.7B and THINC-4B, which demonstrate that small-scale models can achieve high-level reasoning capabilities when trained with this specific paradigm.

Experimental Results

The models were evaluated on five competition-level math benchmarks: AIME 2024–2026, HMMT 2025, and BeyondAIME.

Performance:
- THINC-4B achieved an average accuracy of 78.1%, outperforming all TIR baselines (including ASTER-4B, ReTool-32B, and rStar2-Agent-14B) and surpassing the much larger Qwen3-235B-A22B-Thinking (75.2%) on four of the five benchmarks.
- THINC-1.7B reached 42.8% average accuracy, a 10.6 percentage point improvement over its base model (Qwen3-1.7B) and significantly outperforming other small-scale TIR baselines.
Efficiency: THINC-4B required fewer tool calls per trajectory (6.1 vs. 11.1 for ASTER-4B) and produced shorter responses, indicating higher efficiency.
Robustness: In experiments measuring recovery from initial code execution failures (Recovery@k), THINC-4B maintained a recovery rate of 64–69% even after three consecutive code failures. In contrast, interleaved TIR baselines degraded sharply, with some dropping to near 0% recovery.
Code Grounding: Analysis revealed that 99.2% of THINC-4B's final answers were grounded in interpreter output, compared to significantly lower rates for baselines (e.g., 88.4% for ReTool, 74.3% for rStar2).

Significance

The paper claims that THINC demonstrates that code-centric reasoning is a superior paradigm for mathematical problem solving compared to interleaved TIR. By restricting natural language to high-level planning and forcing all derivations into executable code, the framework eliminates the propagation of arithmetic errors and ensures that the final answer is verified by the interpreter. The results suggest that this structural change allows smaller models (4B parameters) to outperform significantly larger natural-language-only reasoners (235B parameters) and established tool-integrated agents, highlighting the efficacy of "thinking in code" over "thinking with code."