Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning

This paper demonstrates that large-scale pretrained Vision-Language-Action models exhibit remarkable resistance to catastrophic forgetting during continual learning, often achieving zero forgetting with simple experience replay and enabling rapid skill recovery through fine-tuning, a capability that fundamentally differs from smaller models trained from scratch.

The Gist

Here is an explanation of the paper "Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning," broken down into simple concepts with creative analogies.

The Big Problem: The "Goldfish" Robot

Imagine you are teaching a robot to do chores.

1. First, you teach it to make coffee. It learns perfectly.
2. Then, you teach it to fold laundry.
3. Then, you teach it to wash dishes.

In the old days of robotics (using small models trained from scratch), the robot suffered from Catastrophic Forgetting. It was like a goldfish with a 3-second memory. As soon as you taught it to wash dishes, it completely forgot how to make coffee. To fix this, scientists had to use "Experience Replay"—essentially forcing the robot to re-read its old textbooks (the data from previous tasks) every time it learned something new. But this required massive libraries of old data, which is expensive and hard to manage.

The New Discovery: The "Super-Student" Robot

This paper introduces a new type of robot brain called a Vision-Language-Action (VLA) model. These are "pretrained," meaning they have already read the entire internet, watched millions of videos, and learned general concepts about the world before they ever met a specific robot arm.

The researchers asked: "If we teach these super-smart, pre-trained robots new skills one by one, do they still forget the old stuff?"

The Answer: Surprisingly, NO. They barely forget anything at all.

The Key Findings (With Analogies)

1. The "Tiny Notebook" Effect

• The Old Way: To stop a small robot from forgetting, you needed a massive library of old data (like a 20% chunk of its entire history) to review every time it learned something new.
• The New Way: The pre-trained VLA models are so smart that they can learn new skills while only keeping a tiny notebook of old data (as small as 2% of the history).
• The Analogy: Imagine a small student trying to pass a math test while learning physics. They need to re-read their entire math textbook every night to remember algebra. But a genius student who already understands the principles of math only needs to glance at a single sticky note to remember how to solve equations while learning physics. The pre-trained model is the genius student.

2. The "Muscle Memory" of Knowledge

The researchers found that even when the robot's performance on an old task seemed to drop (it looked like it was forgetting), the knowledge wasn't actually gone. It was just "dormant."

• The Analogy: Think of riding a bike. If you haven't ridden one in 10 years, you might wobble and fall at first. It looks like you forgot how. But if you get back on, you don't need to relearn from scratch; you just need a few minutes of practice to "wake up" the muscle memory.
• The Paper's Proof: When they took a robot that seemed to have forgotten a task and gave it just a tiny bit of extra training (finetuning), it instantly remembered the skill perfectly. The knowledge was still there, hidden deep inside its "brain."

3. The "Universal Translator" vs. The "Specialist"

• Small Models (The Specialist): These are like a person hired specifically to stack red blocks. If you ask them to stack blue blocks, they get confused and forget how to stack red ones. They are too specialized.
• Pre-trained VLAs (The Universal Translator): These models are like a polyglot who speaks 50 languages. When they learn a 51st language, they don't forget the first 50. They use their deep understanding of how language works to adapt quickly. Because they already understand the "grammar" of the physical world (vision, language, and movement), adding a new task is just a small tweak, not a total overhaul.

Why Does This Matter?

This changes the rules of robotics forever.

1. Simpler Training: We don't need complex, expensive algorithms to prevent robots from forgetting. We just need to give them a good pre-training and a tiny bit of old data to review.
2. Cheaper Robots: We don't need massive servers to store terabytes of old robot data. A small memory buffer is enough.
3. Real-World Lifelong Learning: This brings us closer to robots that can live in our homes for years, learning new chores every day without needing to be "reset" or retrained from scratch every time they learn a new trick.

The Bottom Line

The paper discovers that big, pre-trained brains are naturally better at remembering things than small, trained-from-scratch brains. They are resilient, efficient, and surprisingly good at keeping their past skills alive while learning new ones. It turns out that in the world of AI, being "well-read" (pretrained) is the best way to avoid forgetting.

Technical Summary

Here is a detailed technical summary of the paper "Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning."

1. Problem Statement

Continual Learning (CL) in robotics requires a policy to acquire new skills sequentially over time without catastrophically forgetting previously learned behaviors. This presents a fundamental stability-plasticity trade-off: the model must be plastic enough to learn new tasks but stable enough to retain old ones.

• Current State: Prior research has focused heavily on small, non-pretrained Behavior Cloning (BC) models trained from scratch. In these settings, catastrophic forgetting is pervasive, often requiring large replay buffers (storing significant amounts of past data) or complex regularization techniques (e.g., EWC) to mitigate.
• The Gap: The behavior of modern, large-scale pretrained Vision-Language-Action (VLA) models (e.g., GR00T, Pi0) in continual learning settings remains underexplored. It is unclear if the representational richness of large pretraining changes the dynamics of forgetting and transfer.

2. Methodology

The authors conducted a comprehensive empirical study comparing pretrained VLAs against small, non-pretrained models across the LIBERO benchmark suite (four task suites: Spatial, Object, Goal, and 10).

• Models Compared:
  • Pretrained VLAs: GR00T N1.5 (NVIDIA) and Pi0 (Black et al.). These are large models (billions of parameters) pretrained on massive internet-scale image-text data and robot trajectories.
  • Non-Pretrained Baselines: BC-Transformer, BC-Diffusion Policy, and BC-ViT. These are smaller models trained from scratch or with limited pretraining.
• Experimental Setup:
  • Task Sequence: Models were trained sequentially on 10 tasks within a suite.
  • Experience Replay (ER): The primary CL method used. A replay buffer stores a subset of transitions from previous tasks. The study varied buffer sizes significantly (0.2%, 2%, and 20% of the total dataset per task).
  • Ablation Studies: To isolate the role of pretraining, the authors compared:
    1. Pi0 pretrained on VL + Action data.
    2. Pi0 initialized from a Vision-Language (VL) backbone (PaliGemma) but without robot action pretraining.
    3. Pi0 architecture trained entirely from scratch.
  • Knowledge Probing: To investigate what is forgotten, the authors used component swapping (swapping the Vision-Language backbone vs. the Action head between training stages) and rapid recovery tests (finetuning on old tasks with minimal steps).

3. Key Contributions & Findings

A. Surprising Resistance to Forgetting

The most significant finding is that pretrained VLAs are remarkably resistant to forgetting compared to small models.

• Zero Forgetting with Small Buffers: While small BC models suffer severe forgetting (Negative Backward Transfer, NBT, > 0.4) with small replay buffers (2% data), pretrained VLAs (Pi0, GR00T) achieve near-zero or even negative NBT (indicating positive backward transfer) with the same small buffers.
• Positive Backward Transfer: In some cases, learning new tasks actually improved performance on previous tasks for pretrained models, challenging the traditional view that learning new skills degrades old ones.

B. The Critical Role of Pretraining

The authors demonstrated that large-scale pretraining is the primary driver of this robustness, not just model size.

• Pretraining vs. Size: Models trained from scratch (even with the same architecture as Pi0) suffered high forgetting. However, initializing with a pretrained VL backbone significantly reduced forgetting, even without robot-specific pretraining.
• Data Efficiency: Pretrained models maintain high forward transfer (learning new tasks) while preserving old knowledge, even with extremely small replay buffers (e.g., 10 samples per task). Non-pretrained models require much larger buffers (>20%) to achieve comparable stability.

C. Knowledge Retention vs. Performance Degradation

The paper reveals a nuanced dynamic: Performance degradation does not equal knowledge loss.

• Latent Knowledge: Even when a VLA's success rate on a past task drops (appearing to "forget"), the underlying task-relevant knowledge remains encoded in the model's internal representations (specifically the Vision-Language backbone).
• Rapid Recovery: When these "forgotten" tasks are revisited via finetuning, pretrained VLAs recover peak performance in a fraction of the time (often <10% of original training steps) compared to non-pretrained models, which must relearn from scratch.
• Compartmentalization: The study found that forgetting is not monolithic. The Vision-Language backbone is the dominant source of performance drop when swapped, suggesting that action mappings are relatively stable, but the semantic understanding of the task degrades slightly, yet remains recoverable.

4. Results Summary

Metric	Non-Pretrained (BC-Transformer)	Pretrained VLA (Pi0/GR00T)
Negative Backward Transfer (NBT)	High (0.2 – 0.5) with small buffers.	Near Zero or Negative (0.0 – -0.1) with small buffers.
Replay Buffer Efficiency	Requires >20% data to minimize forgetting.	Effective with <2% data; often zero forgetting at 2%.
Recovery Speed (Finetuning)	Requires ~100% of original training steps to recover.	Requires <10% of original training steps to recover.
Forward Transfer	Often compromised by stability constraints.	Maintains high success rates on new tasks while preserving old ones.

5. Significance and Implications

• Paradigm Shift in CL: The paper suggests that for large-scale pretrained VLAs, the classical "stability-plasticity" trade-off is fundamentally altered. The need for complex CL algorithms (like EWC or massive replay buffers) may be obviated by the representational capacity of pretraining.
• Practical Robotics: This implies that robots equipped with large VLAs can learn new skills continuously in the real world with minimal memory storage requirements, making lifelong learning more feasible and scalable.
• Mechanism of Forgetting: The finding that knowledge is "latent" and recoverable suggests that forgetting in VLAs is often a matter of retrieval or activation rather than permanent erasure of weights. This opens new avenues for research into "retrieval-based" continual learning rather than just "prevention-based" learning.
• Future Directions: The authors propose that future CL research for robots should focus on reusing retained representations and leveraging pretraining, rather than solely relying on larger replay buffers or specialized regularization.

In conclusion, the paper establishes that pretraining fundamentally changes the dynamics of continual learning, enabling large VLAs to acquire new skills over time with simple experience replay and negligible forgetting, a capability previously unseen in smaller, scratch-trained models.