💻 computer science

Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning

Remember-R1 is a reinforcement learning framework that mitigates long-context visual forgetting in multimodal large language models by applying process-level supervision to encourage sustained visual evidence usage throughout complex reasoning trajectories.

Original authors: Jianmin Chen, Jiaqi Tang, Wei Wei, Xiaogang Xu, Jiafei Wu, Zhe Liu, Qianzhou Wang, Yingying Yan, Botong Geng, Yuyang Xia, Lei Zhang, Qifeng Chen

Published 2026-08-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jianmin Chen, Jiaqi Tang, Wei Wei, Xiaogang Xu, Jiafei Wu, Zhe Liu, Qianzhou Wang, Yingying Yan, Botong Geng, Yuyang Xia, Lei Zhang, Qifeng Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a super-smart robot to solve a mystery by looking at a single, complex picture. You tell the robot, "Look at this image and tell me what's happening." At first, the robot is a detective on high alert, staring intently at every detail in the photo. But as the robot starts talking, writing out its long, step-by-step thoughts, something strange happens. The longer it talks, the more it seems to forget the picture entirely. It starts relying on its own previous words, guessing what might be in the image based on how stories usually go, rather than what is actually there. This is a problem for a new generation of "Multimodal Large Language Models" (MLLMs)—AI systems that can see and read. They are getting better at complex reasoning, but they have a glitch: as their "chain of thought" gets longer, they suffer from "visual forgetting." They drift away from the visual evidence, leading to wrong answers even when the picture holds the truth. Scientists care about this because if these AI models can't keep their eyes on the prize while they think, they can't be trusted for serious tasks like solving math problems with diagrams or understanding medical scans.

Enter Remember-R1, a clever new training method designed to keep the robot's eyes glued to the picture, no matter how long the story gets. Think of the AI's reasoning process as a long road trip. In the past, researchers tried to fix the forgetting problem by constantly stopping the car to show the driver the map again (re-introducing the image), which was slow and clunky. Others tried to ask the driver to make up a list of things they thought they saw and check that list later, but that was like checking a proxy note instead of the actual road. Remember-R1 takes a different approach. Instead of changing the trip or adding stops, it acts like a strict but helpful coach riding in the back seat, whispering rewards to the driver while they are driving.

The paper proposes a system called Remember-R1 that uses a technique called Reinforcement Learning to train these models. The core idea is to supervise the model's "thought process" directly, rather than just grading the final answer. The researchers designed three specific "rewards" to encourage the model to stay grounded in the image:

The "Keyword Hunter" Reward: The model gets points for explicitly mentioning specific, annotated details it found in the image (like "blue sphere" or "tiny cube") throughout its reasoning. It's like a game where the driver gets a bonus for spotting and naming every specific landmark they pass, ensuring they aren't just making things up.
The "Memory Keeper" Reward: This reward punishes the model if it starts paying less attention to the image as the conversation gets longer. The goal is to keep the "visual attention" steady from the first step to the last, preventing the model from drifting into a daydream where it relies only on its own text.
The "Spotlight" Reward: This ensures the model isn't just looking at the image randomly; it must focus its attention on the specific parts of the picture that actually answer the question. If the question is about a red car, the model gets rewarded for keeping its "spotlight" on the red car, not the background trees.

The authors tested Remember-R1 on two different sizes of AI models (3 billion and 7 billion parameters) across seven different benchmarks, including math, logic, and general visual understanding. The results suggest that this method works. The models trained with Remember-R1 consistently performed better than their untrained counterparts and other existing methods. For example, on the MathVista benchmark, the 3B model improved its score from 51.90 to 65.50, and the 7B model jumped from 62.30 to 69.80.

Crucially, the paper shows that this improvement isn't just about getting the right answer at the end; it's about how the model gets there. By analyzing the models' "attention," the researchers found that Remember-R1 slows down the rate at which the model forgets the image. While standard models tend to lose interest in the picture halfway through their reasoning, Remember-R1 models keep their gaze fixed on the visual evidence much longer. The paper explicitly rules out the idea that simply re-showing the image to the model during the process is the best solution, noting that it is too expensive and breaks the flow of reasoning. Instead, they argue that training the model to maintain its own visual focus through these specific rewards is the more effective path.

In short, Remember-R1 suggests that by rewarding AI models for being good "visual detectives" throughout their entire thought process—spotting keywords, keeping their memory fresh, and focusing on the right spots—we can stop them from forgetting what they are looking at. This doesn't just make them smarter at math or logic; it makes them more reliable observers of the world, ensuring that their long, complex stories are always rooted in the truth of the picture they are seeing.

Technical Summary: Remember-R1

Problem Statement

Multimodal Large Language Models (MLLMs) are increasingly utilizing long chain-of-thought (CoT) reasoning for complex tasks. However, a critical weakness identified in current models is long-context visual forgetting. As reasoning sequences lengthen, models tend to rely less on visual evidence and more on the accumulated textual context generated in previous steps. This leads to a drift where the reasoning chain diverges from visual facts, particularly in multi-step mathematical, logical, and spatial reasoning tasks.

Existing mitigation strategies fall into two categories, both with limitations:

Visual Re-introduction: Re-inserting the image or regions into the context during inference. While effective, this increases inference costs (computation and memory) and can fragment the reasoning process.
Visual Claim Proxies: Inserting intermediate visual claims during training. This introduces a "proxy-task gap," as supervision is applied to auxiliary interactions rather than the original reasoning trajectory where forgetting actually occurs.

There is a lack of methods that provide direct supervision on the original reasoning trajectory to constrain how visual evidence is used and maintained without altering the inference procedure.

Methodology: Remember-R1

The authors propose Remember-R1, a Reinforcement Learning (RL) framework designed to mitigate visual forgetting by applying process-level supervision directly to the target reasoning rollout. The framework operates without modifying the inference procedure.

1. Data Annotation

To enable process-level supervision, the authors augment the ViRL39K dataset with structured visual evidence annotations:

Visual Keyword Extraction: Using Qwen-VL-Max, they extract a set of short, grounded lexical units (objects, attributes, quantities, spatial phrases) from the image.
Key-Region Box Annotation: They identify specific image regions (bounding boxes) relevant to answering the question.
Verification: Human verification ensures the quality of keywords and boxes. The final dataset contains 38,657 annotated samples.

2. Reward Design

Remember-R1 decomposes "sustained reliance on visual evidence" into three complementary aspects, each corresponding to a specific process-level reward:

Visual Vocabulary Reward ( $r_{voc}$ ): Encourages the explicit mention of annotated visual keywords throughout the reasoning trace.
- Mechanism: Computes a clipped average of position-weighted matched keywords. It rewards not just the presence of keywords, but their appearance in later steps of the generation, preventing the model from mentioning visual facts only at the beginning.
Visual Memory Reward ( $r_{mem}$ ): Encourages the persistence of visual attention into later reasoning steps.
- Mechanism: Measures the aggregate attention mass assigned to visual tokens ( $\Omega(t)$ ) at generation step $t$ . The reward is based on the difference between the average attention in the final window of generation versus the initial window. It penalizes significant decay in visual attention as the response lengthens.
Visual Key-Region Reward ( $r_{region}$ ): Encourages sustained attention specifically on question-relevant image regions.
- Mechanism: Calculates the ratio of attention assigned to annotated key-region boxes relative to total visual attention. It uses a temporally weighted average, assigning higher weight to later steps to ensure the model remains focused on relevant regions throughout the reasoning process.

3. Reinforcement Learning Optimization

The total reward combines the standard answer correctness reward ( $r_{acc}$ ) with the three visual process rewards:
$R(Y, I, Q) = r_{acc}(Y) + r_{voc}(Y) + r_{mem}(Y) + r_{region}(Y)$
The policy is optimized using Group Relative Policy Optimization (GRPO), sampling groups of responses to compute normalized advantages.

Key Contributions

Process-Level Supervision Framework: The paper proposes a novel approach to studying visual forgetting by directly supervising the original reasoning rollout, avoiding the inference costs of re-introduction and the proxy gaps of claim-based methods.
Three-Aspect Reward Design: The authors design three specific rewards that target the coverage of visual keywords, the persistence of visual attention over time, and the localization of attention to question-relevant regions.
Empirical Validation: The method is validated across two model scales (3B and 7B) and seven diverse benchmarks, demonstrating consistent improvements in reasoning, general multimodal capability, and visual perception.

Experimental Results

The authors evaluated Remember-R1 on Qwen2.5-VL-3B and Qwen2.5-VL-7B base models across seven benchmarks:

Mathematical & Logical Reasoning: MathVision, MathVista, LogicVista.
General Multimodal Capability: MMVet, MMMB, MMStar.
Visual Perception: RealWorldQA.

Key Findings:

Performance Gains: Remember-R1 consistently outperforms the base models and recent reasoning-oriented MLLMs (e.g., Ocean-R1, VLAA-Thinking) and prior visual-forgetting mitigation methods (e.g., DeepSketcher, TVC). Notable improvements were observed on MathVista (+13.60 for 3B, +7.50 for 7B) and MMVet.
Visual Grounding Analysis: Attention ratio analysis shows that while all models exhibit a decline in visual attention over time, Remember-R1 demonstrates a significantly slower decline, particularly in the middle and later stages of generation.
Ablation Studies: Removing any of the three process-level rewards resulted in performance degradation across benchmarks.
- Removing $r_{voc}$ led to fewer visual keyword mentions and weaker reasoning.
- Removing $r_{mem}$ resulted in faster attention decay.
- Removing $r_{region}$ reduced attention to relevant areas, causing accuracy drops across all task types.
Comparison with Accuracy-Only RL: A variant trained only with answer correctness ( $r_{acc}$ ) showed improvements but was inferior to the full Remember-R1, indicating that final-answer supervision alone is insufficient for maintaining visual grounding in long contexts.

Significance and Claims

The paper claims that Remember-R1 effectively mitigates long-context visual forgetting by directly supervising the use of visual evidence on the trajectory where forgetting occurs. The significance lies in demonstrating that:

Process-level reinforcement learning can improve reasoning performance without altering the inference procedure or increasing inference costs.
Sustaining reliance on visual evidence is not just about mentioning keywords, but about maintaining attention persistence and focusing on relevant regions throughout the generation process.
These improvements generalize across different model scales and task types, suggesting that direct supervision of the reasoning trajectory is a promising direction for long-horizon multimodal reasoning.

The authors conclude that their approach helps models better maintain reliance on visual evidence as generation unfolds, addressing a fundamental limitation in current MLLMs without the computational overhead of previous methods.