Information-Theoretic Constraints for Continual Vision-Language-Action Alignment

1. Problem Statement

The paper addresses the critical challenge of catastrophic forgetting in Vision-Language-Action (VLA) models when deployed in open-ended robotic environments requiring continual learning.

• The Core Issue: While VLA models excel at zero-shot generalization, sequential training on new tasks causes performance on old tasks to degrade sharply.
• Root Cause Identification: The authors argue that this degradation is not merely a loss of task-specific weights but a deterioration of cross-modal information structure. As the model adapts to new tasks, the conditional dependencies between visual observations, language instructions, and actions progressively diffuse.
  • Evidence: Attention maps show that high-response regions shift from task-relevant objects to irrelevant background areas, and the alignment between vision and language becomes fragmented.
• Limitations of Existing Methods:
  • Experience Replay (ER): Focuses on marginal data recovery but fails to preserve fine-grained cross-modal relationships.
  • Parameter Regularization (e.g., EWC): Too rigid to handle the non-stationary semantic shifts inherent in VLA alignment.
  • Architectural Expansion: Avoids interference but isolates tasks, preventing the mixed-modal dependencies required for generalized behavior.

2. Methodology: Info-VLA

The authors propose Info-VLA, an information-theoretic framework designed to preserve cross-modal structural stability while allowing behavioral plasticity. The framework operates on a pre-trained VLA policy (specifically $\pi_{0.5}$) and updates only the action head while keeping the visual-language encoder frozen.

The method utilizes two complementary constraints:

A. Replay Anchor Contrastive Learning (RAC)

• Goal: To prevent new task representations from overwriting the structural organization of historical representations.
• Mechanism:
  • A frozen teacher model (the model state after the previous task) generates stable "anchor" representations for replayed samples.
  • The student model (current training) is trained to align its representations of the same replayed samples with these anchors.
  • Contrastive Objective: It pulls positive pairs (student and teacher representations of the same trajectory) closer while pushing negative pairs (representations of different trajectories, including new task samples) apart.
  • Effect: This establishes a stable visual latent space and prevents representational interference between old and new tasks.

B. Cross-Modal Mutual Information Maximization (CMI)

• Goal: To explicitly preserve the statistical dependencies (joint distribution) between visual and language representations, which pairwise contrastive learning alone cannot fully capture.
• Mechanism:
  • Maximizes the Mutual Information (MI) between the teacher's latent representations ($Z^{old}$) and the student's representations ($Z^{new}$).
  • Loss Function: Uses Kullback-Leibler (KL) divergence to minimize the distance between the joint distribution $p(Z^{old}, Z^{new})$ and the product of marginals.
  • Marginal Consistency: Adds a regularization term ($\mathcal{L}_{MC}$) to ensure the global behavioral-intention distributions of the student and teacher remain consistent, preventing degenerate solutions.
  • Effect: Ensures that the semantic structure and hierarchical relationships encoded in the teacher's cross-modal alignment are transferred to the student, maintaining the "shape" of the dependency space.

Overall Training Objective

The total loss function combines the standard flow-matching loss ($\mathcal{L}_{CL}$) with the proposed constraints:
$$\mathcal{L} = \mathcal{L}_{CL} + \lambda_1 \mathcal{L}_{\text{RAC}} + \lambda_2 \mathcal{L}_{\text{CMI}}$$

3. Key Contributions

1. Problem Diagnosis: Identified the structural distortion of multimodal dependencies (diffusion of cross-modal attention) as the fundamental cause of catastrophic forgetting in VLA continual learning, rather than just parameter interference.
2. Novel Framework (Info-VLA): Proposed a unified framework combining Replay-Anchored Contrastive Learning (for stable reference) and Cross-Modal Mutual Information Distillation (for structural dependency preservation).
3. State-of-the-Art Performance: Demonstrated that Info-VLA significantly outperforms existing continual learning baselines in both task retention and adaptation efficiency.

4. Experimental Results

The method was evaluated on the LIBERO benchmark (specifically LIBERO-Long and LIBERO-Goal), which involves sequential learning of robotic manipulation tasks.

• Performance Metrics: Evaluated using Average Accuracy (AA), Forward Transfer (FWT), Negative Backward Transfer (NBT), and Final Average Accuracy (FAA).
• Key Findings:
  • LIBERO-Long (B5-5N1): Info-VLA achieved an AA of 73.6% and Old-task accuracy of 71.0%, significantly outperforming the best baseline (Experience Replay, which scored 66.6% AA and 64.0% Old).
  • LIBERO-Goal (B0-5N1): Info-VLA achieved 73.3% AA, outperforming ER (67.2%) and EWC (43.0%).
  • Stability-Plasticity Trade-off: Info-VLA showed superior retention of old tasks (low NBT) while maintaining high adaptability to new tasks.
  • Ablation Studies:
    • Removing RAC or CMI individually degraded performance, confirming both are necessary.
    • Visualizations of attention maps confirmed that Info-VLA maintains the original compact attention patterns on relevant objects, whereas baselines (like ER) showed significant attention diffusion.

5. Significance

This work provides a theoretical and practical solution to the "stability-plasticity dilemma" in large-scale multimodal robotics. By shifting the focus from simple parameter regularization to information-theoretic structural preservation, Info-VLA enables robots to learn new skills continuously without losing the critical alignment between what they see, what they are told, and what they do. This is a crucial step toward deploying VLA models in real-world, open-ended environments where lifelong learning is mandatory.