🤖 machine learning

Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models

This paper introduces RECAP, an end-to-end replay strategy with dynamic objective reweighting that effectively mitigates general-capability forgetting in large reasoning models trained with reinforcement learning while simultaneously enhancing reasoning performance through flexible reward trade-offs.

Original authors: Hoang Phan, Xianjun Yang, Yuanshun Yao, Jingyu Zhang, Shengjie Bi, Xiaocheng Tang, Madian Khabsa, Lijuan Liu, Deren Lei

Published 2026-06-19

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Hoang Phan, Xianjun Yang, Yuanshun Yao, Jingyu Zhang, Shengjie Bi, Xiaocheng Tang, Madian Khabsa, Lijuan Liu, Deren Lei

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Specialist" Trap

Imagine you hire a brilliant, all-around chef who can cook anything: they can bake a perfect cake, chop vegetables with precision, and even fix a leaky faucet in the kitchen (a metaphor for a model's ability to handle math, vision, and general knowledge).

Then, you decide to train this chef specifically to win a high-stakes pie-eating contest. You put them in a room with only pie recipes and strict rules on how to hold the fork.

What happens?
After a few weeks of intense training, the chef becomes a world-class pie eater. They follow the fork rules perfectly. However, because they spent all their time practicing pie, they start to forget how to chop vegetables or fix the faucet. If you ask them to chop an onion, they might stare blankly or try to eat it whole.

In the world of AI, this is exactly what is happening. Researchers are using a technique called RLVR (Reinforcement Learning with Verifiable Rewards) to teach Large Language Models (LLMs) how to "reason" (solve math problems, follow complex logic). While the models get amazing at reasoning, they start to forget their other skills, like recognizing objects in a picture, reading text in an image, or staying safe and honest.

The Paper's Solution: "RECAP"

The authors propose a new method called RECAP (Replay-Enhanced CApability Preservation). Think of RECAP as a smart training schedule for that chef.

Instead of just feeding the chef pie recipes all day, RECAP does two things:

Replays the Basics: It occasionally brings back the old "vegetable chopping" and "faucet fixing" recipes so the chef doesn't forget them.
Dynamic Weighting: It acts like a smart coach who watches the chef's progress in real-time.

How the "Smart Coach" Works (The Analogy)

Imagine the chef is practicing three things at once:

The Fork Rule (Format): How to hold the fork.
The Taste (Accuracy): Does the pie taste good?
The Cleanup (General Skills): Keeping the kitchen tidy.

In a normal training session, the coach might say, "Practice all three equally!" But that's inefficient.

The Fork Rule is easy. The chef masters it in 5 minutes. If the coach keeps making them practice it for an hour, it's a waste of time.
The Taste is hard. The chef struggles with it.
The Cleanup is getting messy because the chef is distracted.

RECAP's Coach looks at the data and says:

"Hey, the chef has already mastered the Fork Rule. Let's stop focusing on that (lower the weight). Let's spend more time on the Taste and the Cleanup because those are the ones that are still struggling or getting unstable."

RECAP automatically detects which skills are "saturated" (already learned) and which ones are "volatile" (struggling or changing fast). It shifts the training focus to the struggling skills so the model doesn't over-learn the easy stuff while forgetting the hard stuff.

What They Found (The Results)

The researchers tested this on powerful AI models (specifically the Qwen2.5-VL family). Here is what they discovered:

The "Forgetting" is Real: When they trained models only on reasoning tasks, the models got better at math but got significantly worse at things like reading text in images or recognizing objects.
RECAP Saves the Day: By using their dynamic schedule, the models kept their reasoning skills high (sometimes even improving them) but did not lose their general skills. In fact, they often performed better on general tasks than models trained with standard methods.
Efficiency: The models trained with RECAP didn't just get smarter; they also became more efficient. They started giving shorter, more direct answers instead of rambling on, because the system stopped forcing them to "think" (write long chains of text) on tasks that didn't need it.

The Takeaway

The paper argues that we shouldn't just force AI models to become "reasoning machines" by ignoring everything else. If we do, they become one-trick ponies that forget how to be helpful in the real world.

RECAP is a simple, plug-in tool that acts like a balanced diet for AI training. It ensures that while the model learns to solve complex puzzles, it doesn't forget how to see, read, and understand the world around it. It's about keeping the AI well-rounded while still making it a genius at its job.

Technical Summary: Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models

Problem Statement

Reinforcement Learning with Verifiable Rewards (RLVR) has become a standard post-training paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs) and Vision-Language Models (VLMs). However, the authors identify a critical failure mode: general-capability forgetting. While RLVR effectively improves performance on specific reasoning tasks (e.g., mathematics, logical deduction), it often leads to a significant regression in foundational skills acquired during pretraining, such as visual perception, factual grounding, and robustness.

Empirical observations in the paper confirm that open-source reasoning models frequently underperform their base counterparts on non-reasoning benchmarks (e.g., image aesthetics, spatio-temporal reasoning, OCR, and robustness to distribution shifts). The authors note that existing mitigation strategies, such as KL-divergence regularization, are insufficient because they are computed on the current task and do not guarantee the preservation of broader, heterogeneous knowledge. Furthermore, standard experience replay across heterogeneous domains faces the challenge of objective imbalance: it is non-trivial to determine the optimal weighting between reasoning rewards and general capability losses, as different objectives converge at vastly different rates.

Methodology: RECAP

To address these issues, the authors propose RECAP (Replay-Enhanced CApability Preservation), an end-to-end training strategy that integrates general-capability data into the RLVR pipeline with a dynamic objective reweighting mechanism.

Core Components

Replay Strategy: RECAP interleaves data from general domains (e.g., object detection, segmentation, OCR) with target reasoning data during fine-tuning. This ensures the model is continuously exposed to non-reasoning tasks to prevent catastrophic forgetting.
Dynamic Reweighting Mechanism: Instead of using static or manually tuned weights, RECAP adapts the loss coefficients online based on the convergence behavior and instability of individual objectives.
- Convergence Rate ( $c_k$ ): Measured by comparing the average loss of the current sliding window against the previous window. A ratio near 1 indicates saturation (the objective has been learned), while a ratio $>1$ indicates recent improvement.
- Instability ( $i_k$ ): Measured as the coefficient of variation (standard deviation relative to the mean) within the sliding window. High instability suggests the objective is volatile and requires more focus.
- Priority Score: The priority score for objective $k$ is defined as $s_k = c_k + i_k$ .
- Softmax Reweighting: The weights $\lambda_k$ are computed using a temperature-controlled softmax over the priority scores:
  $\lambda_k = \frac{K \exp(s_k / T)}{\sum_{j=1}^K \exp(s_j / T)}$
  This formulation, derived from an entropy-regularized priority allocation problem, naturally down-weights saturated, low-variance objectives (like format compliance, which converges quickly) and shifts capacity toward underperforming or high-variance objectives (like reasoning accuracy or perception).

Theoretical Foundation

The paper establishes that the RECAP reweighting rule is the closed-form solution to an entropy-regularized priority allocation problem. This provides a theoretical justification for the softmax-based weighting, ensuring that the allocation avoids degenerate solutions (focusing on a single objective) while prioritizing objectives with high priority scores.

Key Contributions

Systematic Re-evaluation: The authors systematically demonstrate that reasoning-focused fine-tuning consistently causes regression in general capabilities (perception, robustness) across multiple open-source models (Qwen2.5-VL families).
RECAP Algorithm: They introduce a plug-in scheduler that dynamically reweights objectives based on convergence and instability signals. This method requires no auxiliary models, is magnitude-agnostic, and integrates seamlessly into existing RLVR pipelines (specifically GRPO) with negligible overhead.
Performance Gains: The method not only preserves general capabilities but also improves reasoning performance by enabling more flexible trade-offs among in-task rewards, avoiding the over-optimization of easy signals (like format) at the expense of harder signals (like accuracy).

Experimental Results

The authors evaluated RECAP on Qwen2.5-VL-3B and Qwen2.5-VL-7B across two settings: a "RLVR-only" setting and a "Hybrid" setting (combining RLVR with SFT-style training).

Preservation of General Capabilities: In the RLVR-only setting, RECAP preserved or improved performance on perception and robustness benchmarks (e.g., A-OKVQA, VisOnly, OCRBench) where reasoning-only baselines showed significant degradation. For instance, on the LISA segmentation benchmark, reasoning-only fine-tuning dropped performance from 65.13 to 57.58, whereas RECAP maintained a score of 67.24 (surpassing the base model).
Reasoning Performance: RECAP matched or exceeded the reasoning performance of reasoning-only fine-tuning. On the ScienceQA benchmark, RECAP achieved a score of 71.59, outperforming the base model (6.20) and several open-source reasoning baselines.
Efficiency and Conciseness: The dynamic reweighting led to more efficient reasoning. Models trained with RECAP generated significantly shorter rationales (reducing average thinking length from ~67 words to ~27 words on reasoning tasks) without compromising accuracy, thereby reducing inference latency and compute costs.
Comparison to Baselines: RECAP consistently outperformed strong baselines including:
- Reasoning-only: Which suffered severe forgetting.
- Uniform/PropMix: Which used static weighting and failed to adapt to convergence rates.
- LwF (Learning without Forgetting): Which used KL regularization but achieved lower reasoning scores compared to RECAP.
- MoDoMoDo: A complex, computationally expensive baseline that requires proxy models and manual tuning.

Significance and Claims

The paper claims that RECAP offers a principled, low-overhead solution to the "reasoning vs. general capability" trade-off. By shifting the post-training focus away from saturated objectives (like format compliance) toward volatile or underperforming ones (like accuracy and perception), RECAP enables models to "learn to reason" without "unlearning" to perceive.

The authors emphasize that their method is end-to-end and does not require training additional reward models or heavy hyperparameter tuning. They position RECAP as a necessary evolution in post-training pipelines, arguing that optimizing for narrow reasoning rewards without dynamic regularization leads to brittle models that fail in real-world, heterogeneous scenarios. The work suggests that the "forgetting" observed in current reasoning models is not an inevitable side effect of RLVR but a solvable optimization problem through dynamic objective management.