💬 NLP

MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task

The paper introduces MathFimer, a novel framework that enhances large language models' mathematical reasoning by training them to reconstruct missing intermediate steps via a "Fill-in-the-middle" task, thereby generating expanded datasets that improve performance on benchmarks like GSM8K and MATH without relying on powerful external models or high computational costs.

Original authors: Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Xin Xu, Mengdi Zhang, Jian Shao, Yueting Zhuang

Published 2026-02-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Xin Xu, Mengdi Zhang, Jian Shao, Yueting Zhuang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

🧩 The Big Idea: "Fill-in-the-Middle" for Math

Imagine you are teaching a robot how to bake a cake. You give it a recipe that says:

Start: Mix the flour and sugar.
End: Put the cake in the oven.

The robot sees the start and the end, but it has no idea how to get from mixing to baking. It might try to put the raw bowl directly into the oven, or it might forget to add the eggs.

MathFimer is a new tool designed to teach robots (AI models) how to solve math problems by filling in those missing "middle" steps. It doesn't just give the robot the answer; it teaches the robot how to write a detailed, step-by-step story to get there.

🚧 The Problem: The "Black Box" of Math

Current AI models are great at math, but they often learn from "lazy" textbooks. These textbooks might show:

Question: "What is 2 + 2?"
Answer: "4."

Or, they show a solution that jumps from "Let's solve for X" straight to "X = 5," skipping the actual algebra in between.

When AI models learn from these "skipped" steps, they become good at guessing the answer but bad at explaining how they got there. If they make a tiny mistake in a skipped step, the whole answer is wrong, and they don't know why.

The Old Way to Fix This:
Previously, to get better steps, researchers had to hire "super-intelligent" (and very expensive) AI models to write the missing steps for the smaller models. It was like hiring a Nobel Prize-winning mathematician to teach a kindergarten class. It works, but it costs a fortune and takes forever.

💡 The Solution: The "Fill-in-the-Middle" (FIM) Trick

The authors of this paper had a brilliant idea. They looked at how programmers train AI to write code. In coding, there is a game called "Fill-in-the-Middle."

The Game:
You show the AI the beginning of a sentence and the end of the sentence, and it has to guess the middle.

Start: "Once upon a time, there was a..."
End: "...who lived in a castle."
AI's Job: "...little dragon..."

The researchers realized: We can play this same game with math!

How MathFimer Works (The 3-Step Process)

The Setup (The Puzzle):
They took thousands of existing math problems that already had answers. They took a solution that looked like this:
- Step 1: Define the variables.
- Step 2: Set up the equation.
- Step 3: Solve for X.
- Step 4: Check the answer.
Then, they hid Step 2 and Step 3. They gave the AI the "Start" (Step 1) and the "End" (Step 4) and asked: "What goes in the middle?"
The Training (The Practice):
They taught a specific AI model (called MathFimer-7B) to play this game over and over again. It learned to look at the beginning and the end of a math problem and invent the missing logical steps that connect them.
- Analogy: It's like training a bridge builder. You show them the two banks of a river (the start and end of the problem) and ask them to design the bridge (the missing steps) that connects them safely.
The Upgrade (The Expansion):
Once the model learned to play the game, they used it to go back and "fix" old math textbooks. They took existing solutions that were too short and used MathFimer to insert detailed, missing steps.
- Result: A short, lazy answer like "X = 5" became a detailed story: "First, we move the 3 to the other side... then we divide by 2... therefore X is 5."

🏆 Why This is a Big Deal

The paper tested this new method on many different math tests (like the ones used in schools and competitions). Here is what they found:

Better Grades: Models trained on these "expanded" answers got significantly higher scores. They didn't just guess; they understood the logic.
Cheaper & Faster: Unlike the old method that needed super-expensive AI to write the steps, MathFimer can be trained on a standard computer. It's like teaching the robot to teach itself, rather than hiring a human tutor.
No Magic Required: They didn't need to invent new math. They just took existing, correct answers and made them more detailed.

🌟 The "Human" Analogy

Imagine you are learning to drive.

Old Way: Your instructor says, "Turn the wheel, and the car goes left." You memorize the action, but if you encounter a slippery road, you crash because you don't understand why the car turns.
MathFimer Way: Your instructor says, "Turn the wheel. This changes the angle of the tires. The friction against the road pushes the car sideways. Because the car is heavy, it takes a moment to turn. That is why you go left."

MathFimer gives the AI the "friction and physics" of math. It fills in the gap between "I know the answer" and "I know how to get there."

🚀 Conclusion

MathFimer is a clever, low-cost way to make AI smarter at math. By turning math problems into "fill-in-the-blank" puzzles, it teaches AI models to write out their thinking process in full detail. This makes them more reliable, more accurate, and better at solving complex problems without needing expensive super-computers to help them.

1. Problem Statement

Mathematical reasoning is a critical benchmark for Large Language Models (LLMs). While Chain-of-Thought (CoT) prompting has improved performance, the quality of reasoning in training data remains a bottleneck. Existing methods to improve reasoning steps face three major challenges:

Dependency on Larger Models: Current approaches often require even more powerful external models to generate better steps, creating a resource-intensive cycle.
High Computational Cost: Techniques like Monte Carlo Tree Search (MCTS) or repeated inference are computationally expensive.
Reliability Issues: Generating entirely new reasoning chains can introduce hallucinations or errors, whereas existing human-verified solutions are often too coarse-grained (lacking intermediate steps).

The core research question is: Can we efficiently expand reasoning steps in existing, valid solutions to improve model performance without relying on powerful external models or expensive inference procedures?

2. Methodology: MathFimer

The authors propose MathFimer, a framework inspired by the "Fill-in-the-Middle" (FIM) task used in code completion. Instead of generating new solutions from scratch, MathFimer inserts detailed intermediate steps into existing CoT chains.

A. Data Construction (NuminaMath-FIM)

Source: The authors use the high-quality NuminaMath-CoT dataset (853K pairs).
Decomposition: Solutions are decomposed into individual steps.
FIM Transformation: For each problem, a random step $y_i$ is selected as the "Middle" ( $M$ ). The preceding steps form the "Prefix" ( $P$ ), and the succeeding steps form the "Suffix" ( $S$ ).
Format: The data is formatted as $P + \text{<|fim\_prefix|>} + M + \text{<|fim\_middle|>} + S + \text{<|fim\_suffix|>}$ .
Scale: This process generates 2.5 million training samples (3 variations per original problem).

B. Model Training

Base Model: A math-specialized model, Qwen2.5-Math-7B, is used as the foundation.
Objective: The model is trained via Supervised Fine-Tuning (SFT) to predict the missing middle steps ( $M$ ) given the prefix and suffix.
Loss Calculation: Loss is computed only on the tokens following the <|fim_middle|> token.
Result: The trained model, MathFimer-7B, learns to reconstruct missing logical bridges between steps.

C. Step Expansion Pipeline

Once trained, MathFimer-7B is applied to expand existing datasets (e.g., GSM8K, MATH, MetaMathQA):

Inference: For every pair of consecutive steps in a solution, the model infers the missing intermediate reasoning.
Similarity Filtering: To prevent redundancy, the generated step is compared to the original subsequent step. If the similarity exceeds a threshold ( $\eta = 0.8$ ), the generation is discarded as it adds no value.
Insertion: Validated steps are inserted into the original chain, creating a more granular "expanded" dataset.
Iterative Application: The process can be repeated (iterative expansion) to further refine reasoning chains.

3. Key Contributions

Novel Framework: Introduction of the FIM paradigm to mathematical reasoning, shifting from "generating new chains" to "enriching existing valid chains."
Resource Efficiency: The method does not require larger teacher models or complex search algorithms; it relies on a specialized 7B model trained on decomposed data.
Datasets & Model Release: Release of NuminaMath-FIM (2.5M samples) and the MathFimer-7B model.
Scalability: Demonstrated that the method works across different base model sizes (8B to 72B) and can be iterated to further improve performance.

4. Experimental Results

The authors evaluated the approach on multiple benchmarks (GSM8K, MATH, Math Odyssey, OlympiadBench-EN) using various base models (Llama-3.1, Qwen2.5-Math).

Performance Gains: Models trained on MathFimer-expanded data consistently outperformed those trained on original data.
- GSM8K: Up to +7.43% improvement (on Llama-3.1-8B with MathInstruct-CoT).
- MATH: Up to +8.86% improvement (on Qwen2.5-Math-7B with MetaMathQA).
- Average: Consistent gains across elementary and competition-level math tasks.
Iterative Expansion: Multiple rounds of expansion (up to 3 iterations) yielded cumulative improvements, with GSM8K accuracy reaching 83.32% (from 67.78% baseline) on the MathInstruct dataset.
Model Size Analysis: Surprisingly, the performance gap between MathFimer-1.5B, 7B, and 72B was minimal, suggesting that step expansion is a structural task not heavily bottlenecked by model capacity.
Comparison with Other Methods:
- vs. Prompt-based Fill: MathFimer significantly outperformed zero-shot prompting with general models (e.g., Llama-3.2-3B).
- vs. RL/DPO: The method is orthogonal to Reinforcement Learning (RL) and DPO. In fact, using MathFimer-expanded data as a starting point for RL yielded even higher gains than RL on raw data.
Quality Verification: Using a Process Reward Model (PRM), the expanded steps maintained high correctness scores (0.8–1.0 range), and human evaluation confirmed a 94% agreement rate with PRM scores.

5. Significance and Impact

Practicality: MathFimer offers a scalable, cost-effective solution to enhance LLM reasoning without the need for massive compute resources or proprietary "teacher" models.
Data-Centric AI: It highlights that the granularity of reasoning steps in training data is a critical factor for performance, often more so than the sheer volume of data.
Generalizability: While focused on math, the framework showed promise in out-of-distribution (OOD) tasks (logical and scientific reasoning), suggesting the "step expansion" capability is transferable.
Future Direction: The paper suggests that combining step expansion with RL (Process Supervision) creates a powerful synergy, potentially paving the way for more robust reasoning agents.

In conclusion, MathFimer demonstrates that refining the structure of existing knowledge through targeted step insertion is a highly effective strategy for advancing the mathematical reasoning capabilities of LLMs.