💻 computer science

Memory Anchors for Continual Robot Learning

This paper introduces "Memory Anchors," a strategic subset of past experiences identified by conflicting task representations, which, when prioritized in replay buffers, significantly mitigates catastrophic forgetting and enables effective continual learning for robots.

Original authors: Maximilian Du, Zhanyi Sun, Chen Xu, Paarth Shah, Masha Itkina, Shuran Song

Published 2026-08-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Maximilian Du, Zhanyi Sun, Chen Xu, Paarth Shah, Masha Itkina, Shuran Song

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot arm learning to perform a series of tasks, one after another. In the real world, these tasks are rarely isolated; they often look very similar but require different, sometimes opposite, movements. A robot might need to open a jar by turning it clockwise, and then later learn to open a different jar by turning it counter-clockwise. The challenge for artificial intelligence is that when it learns the new skill, it often overwrites the memory of the old one, a phenomenon scientists call catastrophic forgetting. To prevent this, researchers typically use a method called experience replay, where the robot practices on a mix of old and new data, much like a student reviewing past notes while studying for a new exam. For years, the standard approach has been to pick these old notes at random, assuming that any sample from the past is equally helpful.

However, a new study from researchers at Stanford University and the Toyota Research Institute suggests that not all memories are created equal. They discovered that within the vast history of a robot's experiences, there exists a tiny, critical subset of data that acts as a vital anchor, holding the robot's knowledge of previous tasks in place. The researchers call these specific memories "Memory Anchors." Their work shows that if a robot's training data misses even a small fraction of these anchors, the robot forgets its old skills much faster. Conversely, if the training data is deliberately enriched with these specific memories, the robot can learn new, conflicting tasks without losing the old ones. This finding shifts the focus from simply collecting more data to carefully selecting the most important moments from the past to protect the robot's growing intelligence.

The researchers began by observing that even when robots use experience replay, their performance is surprisingly sensitive to exactly which old data they happen to pick for practice. In their experiments, they found that some random selections of old data allowed the robot to retain its skills perfectly, while other random selections caused the robot to forget the same skills almost entirely. This inconsistency pointed to a hidden pattern: certain tasks were much harder to learn sequentially because they shared a visual similarity with previous tasks but demanded a completely different physical action. For instance, a robot might see a bowl and a jar that look similar, but one requires lifting while the other requires twisting. When the robot learns the new task, its internal understanding of the object's appearance can collapse into the same mental category as the old task, causing confusion about which action to take.

To solve this, the team developed a method to identify and isolate these critical moments. They looked for the specific points in the new task where the robot's current understanding of the situation clashed most violently with what it had learned before. They found the moments where the robot's eyes saw something familiar, but its brain knew it had to do something different. From these moments of conflict, they reached back into the robot's past and pulled out the specific old experiences that looked the most similar to the confusing new situation. These retrieved memories are the Memory Anchors. They serve as a reference point, reminding the robot that while the object looks the same, the action required is different, effectively preventing the new learning from erasing the old.

The team tested this idea by deliberately removing these anchors from the robot's training data. The results were stark: when they excluded just ten percent of the best anchors, the robot's forgetting of past tasks increased by more than four and a half times. This proved that a very small number of specific memories were doing the heavy lifting to preserve the robot's history. In a second set of tests, they did the opposite: they took a standard training buffer and filled it with extra Memory Anchors. This simple adjustment reduced the forgetting of difficult, conflicting tasks by sixty-three percent. The robot was able to learn a sequence of tasks that would have otherwise caused it to fail, maintaining its ability to perform the first task even after mastering the third.

To ensure this was not just a simulation, the researchers took their method to a real robot arm in a laboratory. They set up a series of tasks involving identical-looking jars that required different opening strategies: one needed a counter-clockwise twist, another a clockwise twist, and a third a direct lift. Without their special method, the robot would learn the first task, then forget it completely when learning the second, or fail to learn the second because it was too afraid of messing up the first. When they applied the Memory Anchor strategy, the robot successfully learned all three tasks in sequence. It achieved a success rate that was 1.7 times higher than a robot trained with a standard, random mix of old data. The robot learned to distinguish the subtle differences between the jars and execute the correct motion for each, proving that the anchors helped it balance the need to learn new things with the need to remember the old.

The study also explored how this works with different types of robot brains, including large, pre-trained models that are already quite smart. Even for these advanced systems, which are generally better at remembering, the presence of Memory Anchors made a significant difference when the training data was limited. The researchers found that these models, too, suffered from forgetting when the critical anchors were missing, and they improved when the anchors were added. This suggests that the principle is fundamental to how machines learn from experience, regardless of the complexity of the software. The key is not just having data, but having the right data at the right time to act as a stabilizer against the chaos of new learning.

This research offers a new way to think about how machines can grow smarter over time without losing their past. It suggests that the path to a robot that can learn continuously in the wild is not just about feeding it more information, but about teaching it to recognize and value the specific moments where its past and present collide. By identifying these Memory Anchors, engineers can build systems that are more robust, capable of handling the messy, overlapping reality of the physical world. The work moves the field forward by showing that in the journey of continual learning, the quality of the memory matters far more than the quantity, and that a few well-chosen moments from the past can secure the future of a robot's intelligence.

Technical Summary: Memory Anchors for Continual Robot Learning

Problem Statement

Robot policies deployed in real-world environments must possess the capability to continually learn new tasks without suffering from catastrophic forgetting, where learning new behaviors overwrites knowledge of previously learned tasks. A primary challenge in this domain is that new tasks are rarely independent; they often share observational similarities (e.g., similar objects or scenes) with old tasks but require conflicting actions (e.g., turning a jar clockwise vs. counter-clockwise).

While Experience Replay (ER)—training on a buffer of past data alongside new data—is a standard approach to mitigate forgetting, the authors observe that its effectiveness is highly sensitive to data selection. Randomly sampled buffers often fail to preserve performance on specific "high-conflict" task pairs, leading to drastic performance drops despite the presence of past data. The paper posits that a small, critical subset of past experiences plays a disproportionately large role in anchoring past performance, yet these specific memories are not guaranteed to be selected in a random buffer.

Methodology: Memory Anchors and ANCHORER

The authors introduce Memory Anchors, a specific subset of past task data that is critical for preventing catastrophic forgetting during sequential training. They propose a three-step algorithm to identify and extract these anchors, followed by a method called ANCHORER to enrich the replay buffer with them.

1. Identifying Memory Anchors

The extraction process utilizes the current policy's latent space and predictions to find data points where new tasks conflict with old tasks:

Step 1: Find State Representation Overlap. The algorithm maps new task observations ( $D_n$ ) into the latent observation space of the policy trained on past tasks ( $\pi_{n-1}$ ). It identifies a subset of new data ( $D_n^{SID}$ ) where the representations collapse onto those of past tasks, indicating visual or semantic similarity despite task differences.
Step 2: Isolate Action Disagreements. From the overlapping data, the algorithm identifies points requiring conflicting actions. It uses a generative approach: adding Gaussian noise to the ground-truth action of a new data point and denoising it using the old policy ( $\pi_{n-1}$ ). If the denoised action ( $\hat{a}$ ) significantly differs from the ground truth ( $a$ ), the point is flagged as having high action disagreement ( $D_n^{OD}$ ). This step filters out shared sub-trajectories that do not require policy updates.
Step 3: Extract Memory Anchors. The algorithm retrieves the most similar past task data points to the high-disagreement set ( $D_n^{OD}$ ) within the latent space. These retrieved past samples constitute the Memory Anchors ( $D_{Anchor}$ ).

2. The ANCHORER Algorithm

To leverage these findings, the authors propose ANCHORER, an Experience Replay strategy that reserves a specific proportion (e.g., 10% or 20%) of the replay buffer exclusively for these extracted Memory Anchors. This ensures that the most critical regularizing data is present during the training of new, conflicting tasks.

Key Contributions

Memory Anchors Analysis: The paper demonstrates that a small proportion of past data (Memory Anchors) contributes disproportionately to past task retention. Removing just 10% of the best anchors from a replay buffer leads to a 4.5x increase in catastrophic forgetting on the LIBERO benchmark.
ANCHORER Algorithm: A method to enrich ER buffers with Memory Anchors, reducing catastrophic forgetting on high-conflict task pairs by 63% and improving overall task retention on homogeneous task suites.
Real-Robot Validation: The authors construct and validate task suites on a real robot (ARX arm) involving visually similar jars requiring opposing strategies (OpenJar) and language-conditioned sweater folding (SweaterFold). ANCHORER improved total task success by 1.7x compared to a randomly sampled buffer, successfully balancing plasticity (learning new tasks) and stability (retaining old tasks).
VLA Sensitivity: The study extends findings to Vision-Language-Action (VLA) models ( $\pi_{0.5}$ ), showing that even these robust models exhibit significant forgetting on small buffers and are sensitive to the presence of Memory Anchors.

Experimental Results

LIBERO Benchmarks: On the LIBERO-Goal suite (homogeneous tasks), removing the top 10% of Memory Anchors increased forgetting significantly. Conversely, enriching the buffer with 10%–20% Memory Anchors reduced forgetting on the most conflicting task pairs by 63%.
Buffer Size Sensitivity: The benefits of ANCHORER are most pronounced with smaller buffer sizes (e.g., 1% of past data), where random sampling is less likely to capture the critical anchors naturally.
Real-World Performance: In the OpenJar and SweaterFold real-robot experiments, ANCHORER prevented the robot from overgeneralizing or losing plasticity. For instance, in the OpenJar suite, a standard ER buffer failed to learn the second task (Clockwise) without forgetting the first (Counter-clockwise), whereas ANCHORER successfully learned both.
Comparison to Baselines: ANCHORER outperformed other buffer selection methods like Maximal Interference Retrieval (MIR) and Gradient-Based Sample Selection (GSS), as well as an ablation of their own method that omitted the action disagreement step.

Significance and Claims

The paper claims that the mechanism of catastrophic forgetting in robotics is driven by specific task-task conflicts where observation spaces overlap but action strategies diverge. The significance of this work lies in shifting the paradigm of Experience Replay from uniform random sampling to targeted retrieval of critical data.

The authors argue that:

Not all past data is equally valuable for continual learning; a small subset acts as "anchors" that stabilize performance against conflicting new inputs.
Identifying these anchors via action disagreement in overlapping latent spaces is a robust heuristic for selecting data that prevents destructive overwriting.
This approach is essential for homogeneous tasks (same environment/objects, different actions), which represent a realistic and difficult scenario for continual robot learning, as opposed to heterogeneous tasks where environmental changes naturally separate task representations.

The work concludes that while Memory Anchors are naturally present in large, diverse buffers, their explicit identification and concentration are necessary to ensure stability in resource-constrained or highly conflicting continual learning scenarios.