🤖 AI

AdaR: A Framework for Equipping LLMs with Adaptive Reasoning

The AdaR framework enhances large language models' mathematical reasoning robustness and generalization by training them with Reinforcement Learning on automatically synthesized, logically equivalent queries to penalize spurious correlations and encourage adaptive reasoning.

Original authors: Zhejian Lai, Xiang Geng, Zhijun Wang, Yang Bai, Jiahuan Li, Rongxiang Weng, Jingang Wang, Xuezhi Cao, Xunliang Cai, Shujian Huang

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhejian Lai, Xiang Geng, Zhijun Wang, Yang Bai, Jiahuan Li, Rongxiang Weng, Jingang Wang, Xuezhi Cao, Xunliang Cai, Shujian Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of artificial intelligence, large language models have emerged as powerful tools capable of solving complex problems, from writing poetry to debugging code. Among their most impressive feats is mathematical reasoning, where these systems break down a question into a series of logical steps to reach a solution. For years, researchers have celebrated this ability, assuming that when a model solves a problem correctly, it truly understands the logic behind it. However, a closer look reveals a troubling flaw: many of these models are not actually reasoning through the problem. Instead, they are often memorizing patterns from their training data. They learn to associate specific numbers or phrases with certain answers, much like a student who memorizes the answer key for a practice test without understanding the underlying math. When the test changes slightly—when the numbers are different or the wording is tweaked—these models often stumble, failing to adapt because their "reasoning" was never real to begin with.

This is the central challenge addressed by a new study from researchers at Nanjing University and Meituan, who have developed a framework called AdaR to teach machines how to think adaptively. The team discovered that the root of the problem lies in what they call "spurious reasoning." This occurs when a model relies on superficial connections, such as the specific value of a number in a question, rather than the structural logic of the problem itself. To illustrate, imagine a model trained to solve a problem where a number is cubed. If the training data always used the number 3, the model might learn to simply output "cube" whenever it sees a multiplication problem, regardless of the actual numbers involved. If the problem changes to involve a different number, the model fails because it never learned the rule of cubing; it only learned to recognize the pattern of the number 3. The researchers found that even advanced models, which perform brilliantly on standard tests, suffer from this brittleness, unable to generalize their skills to new, slightly different scenarios.

To fix this, the researchers created a system that forces the model to learn the logic rather than the numbers. They started by taking existing math problems and stripping them down to their core structure, or template. They then used a computer program to automatically generate thousands of new versions of these problems by changing the specific numbers while keeping the logical steps exactly the same. Crucially, they did not just ask a language model to guess the new answers; they wrote executable code to calculate the correct answers with absolute certainty. This ensured that every new problem they created was valid and had a verified solution. They then subjected their models to a rigorous training process using these new, varied problems. Instead of simply showing the model the right answer, they used a method that rewarded the model only when it could solve the entire set of varied problems correctly. If the model relied on memorized patterns, it would fail on the new numbers and receive a penalty. If it learned the actual logic, it would succeed across all variations and receive a reward.

The results of this approach were striking. When tested on both familiar and completely new types of math problems, the models trained with this adaptive framework showed significant improvements. On average, their performance jumped by more than eight points compared to other methods, a substantial gain in the world of artificial intelligence. More importantly, the models became much more robust. They could handle changes in the numbers of a problem without breaking down, proving they had learned the underlying rules rather than just memorizing specific examples. The researchers also observed a change in how the models "thought." Before this training, the models tended to focus on the specific numbers in a question, ignoring the structure. After training, their attention shifted to the structure of the problem itself, allowing them to treat unknown variables with the same logical care as known numbers. This shift suggests the models were developing a form of algebraic thinking, treating problems as systems of relationships rather than lists of facts to be recalled.

The study also explored how different types of changes affected the learning process. They found that changing the numbers in the problems was far more effective at teaching the model to reason than simply rewriting the questions in different words. This indicates that the key to adaptive reasoning is exposing the model to a wide variety of numerical scenarios, forcing it to rely on logic to navigate the differences. The researchers noted that while the method worked well, it had limits; it required the problems to be solvable by a computer program, which meant it was best suited for math and logic tasks rather than more abstract fields like proving complex theorems. Furthermore, the success of the method depended on the model already having a solid foundation of mathematical knowledge. The system could not teach reasoning from scratch to a model that knew nothing; it could only refine the reasoning of a model that already had the basic building blocks.

Ultimately, this work offers a clear path forward for making artificial intelligence more reliable. By moving away from simple memorization and toward adaptive reasoning, the researchers have shown that it is possible to build models that can handle the unexpected. The findings suggest that the future of intelligent systems lies not just in making them bigger or faster, but in teaching them to understand the rules that govern the world, rather than just the specific examples they have seen before. As these models become more integrated into our daily lives, from solving financial equations to planning complex logistics, the ability to adapt to new information without failing will be essential. The AdaR framework provides a blueprint for achieving this, turning brittle, pattern-matching machines into flexible, logical thinkers capable of navigating a world that is constantly changing.

Technical Summary: AdaR – A Framework for Equipping LLMs with Adaptive Reasoning

1. Problem Statement

Large Language Models (LLMs) have demonstrated strong capabilities in mathematical reasoning, yet they exhibit significant failures in robustness (performance on in-domain tasks with altered conditions) and generalization (performance on out-of-domain tasks). The authors attribute these deficiencies to spurious reasoning.

In spurious reasoning, models rely on superficial correlations between query templates and reasoning traces rather than establishing correct causal links. Consequently, models tend to blindly reproduce memorized patterns or variable values from training data. For instance, if a model learns that a specific equation structure implies taking a "cube root," it may output "cube root" even when the equation requires a different operation, simply because the superficial pattern matches. This non-compositional reasoning prevents models from adapting to new variable values or query structures, leading to brittle performance.

2. Methodology: The AdaR Framework

To address spurious reasoning, the authors propose AdaR, a framework designed to equip LLMs with adaptive reasoning—the ability to establish correct correlations between a query template and the reasoning process, independent of specific variable values. The framework consists of two primary components: automated data synthesis and a specific training strategy.

2.1 Data Synthesis with Executable Code and Verifiable Answers

Traditional data augmentation methods often struggle to systematically vary both query templates and variable values while maintaining logical consistency. AdaR overcomes this by decomposing data synthesis into verifiable sub-tasks:

Logic Extraction and Code Generation: A strong LLM is prompted to extract the query template ( $T$ ), the variable set ( $x$ ), and the problem-solving logic ( $L_T$ ) from a seed query. Crucially, the logic is translated into executable code (e.g., Python). This code serves as a ground-truth generator for any perturbed query.
Controllable Perturbation: The framework applies independent, controlled perturbations to the variable values in $x$ . Values are sampled within a predefined range ( $\pm \alpha\%$ ) while strictly preserving their numeric type and sign. This ensures the generation of diverse queries ( $q'$ ) that share the same underlying logic but differ in numerical instantiation.
Sanity Check: To ensure data quality, the framework performs three checks:
- Variable Alignment (VA): Verifies that variables in the template match those in the generated code.
- Executable Code (EC): Ensures the code runs without errors and reproduces the original gold answer when given original inputs.
- Existence of Valid Query (EVQ): Cross-validates the code-generated answer against a mathematical LLM to ensure the perturbed query is well-posed and not invalid due to violated constraints.
Paraphrasing: Query templates are also paraphrased to increase diversity in the textual expression of the problem.

The resulting dataset consists of query-answer pairs without Chain-of-Thought (CoT) traces, as the CoT is generated during training.

2.2 Training Strategy: RLVR

The authors employ Reinforcement Learning with Verifiable Rewards (RLVR) rather than standard Supervised Fine-Tuning (SFT) or Rejection Sampling Fine-Tuning (RFT).

Rationale: In single-data settings, spurious reasoning can occasionally yield correct answers by chance, making outcome rewards unreliable. However, when evaluated on a batch of perturbed queries generated by AdaR, spurious reasoning is unlikely to remain valid across all variations.
Mechanism: The model is trained to maximize rewards based on the correctness of answers across these perturbed queries. This forces the model to explore adaptive problem-solving logic that generalizes across variable changes, rather than memorizing superficial patterns associated with specific numbers.

3. Key Contributions

Identification of Spurious Reasoning: The paper formally attributes robustness and generalization failures in mathematical LLMs to the reliance on superficial correlations (spurious reasoning) rather than compositional logic.
The AdaR Framework: A novel pipeline that synthesizes high-quality, logically equivalent data by controllably perturbing variable values and verifying answers via executable code.
Adaptive Reasoning via RLVR: A training strategy that leverages the diversity of synthetic perturbed data to penalize spurious logic and encourage the learning of template-logic correlations.
Comprehensive Evaluation: Extensive experiments demonstrating that AdaR improves performance on both in-domain robustness tasks and out-of-domain generalization tasks using a small amount of synthetic data (9K instances).

4. Experimental Results

Experiments were conducted on three base models: Qwen2.5-MATH-7B, DeepSeekMath-7B, and LLaMA3-8B.

Performance Gains: AdaR achieved substantial improvements across all benchmarks. On average, it outperformed the best baseline (MetaMATH) by 8.50 points and MathGenie by 11.44 points.
Robustness and Generalization:
- In-Domain: Significant gains were observed on perturbed variable values within seen templates (ORCA-AdaR-test) and unseen templates with increasing difficulty (GSM-SYM).
- Out-of-Domain: The method improved generalization on datasets like MATH, CollegeMath, TheoremQA, and AIME 2025. For the Qwen2.5-MATH-7B model, the average gain on out-of-domain tasks reached +8.56 points.
Ablation Studies:
- Sanity Checks: All components (VA, EC, EVQ) were essential. Removing Executable Code (EC) resulted in insufficient valid data (0.2%) for training.
- Training Strategy: Standard RLVR on single data degraded performance compared to SFT due to unreliable rewards. AdaR's multi-data setting was crucial for preventing overfitting to numerical patterns.
- Scaling: Scaling the variable set ( $x$ ) yielded the highest marginal returns, followed by template ( $T$ ) and logic ( $L$ ) scaling.

5. Analysis and Significance

The paper provides several analyses to explain why AdaR works:

Attention Correction: Attention heatmaps reveal that standard SFT causes models to attend to superficial numerical tokens (e.g., specific numbers like "32") rather than the structural template. AdaR corrects this, shifting attention back to the query template, which encodes the true logic.
Emergence of Algebraic Thinking: After training with AdaR, the proportion of CoTs containing structural code snippets (e.g., variable substitution) increased from 55% to 90%, indicating the model learned to treat unknowns and knowns algebraically rather than memorizing arithmetic steps.
Limitations of Advanced Models: Even advanced LLMs exhibit robustness deficiencies that are effectively mitigated by AdaR, suggesting that high performance on standard test sets often masks fundamental generalization flaws.

Significance:
The authors claim that AdaR offers a scalable and broadly applicable framework to mitigate the brittleness of LLMs in mathematical reasoning. By moving away from memorization toward adaptive reasoning, the framework enables models to solve problems based on underlying logical structures rather than superficial patterns. The work highlights that perturbing variable values is a more effective strategy for improving reasoning than merely paraphrasing templates, and that verifiable rewards in a diverse data setting are essential for learning true adaptive logic.

Limitations Acknowledged:
The authors note that their current implementation focuses on scaling $T$ and $x$ , while scaling the diversity of problem-solving logic ( $L$ ) remains an open direction. Additionally, the framework requires seed queries to be expressible as executable programs with perturbed inputs, which may limit immediate application to domains like theorem proving without further design. Finally, computational constraints prevented testing on larger reasoning models.