Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training

Technical Summary: Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training

1. Problem: The "Lock-In" Phenomenon

When generalist Vision-Language-Action (VLA) models are post-trained on small, specialized datasets (low-data regime), they often suffer from a performance-generality dilemma. While they successfully learn the target skill, they become "locked-in" to the specific distribution of the post-training demonstrations. The authors formalize this failure into two distinct categories:

• Concept Lock-In: The policy loses its ability to ground language to different objects or attributes. It fixates on the specific object identities (e.g., "blue block") seen during training and ignores novel instructions (e.g., "green block").
• Spatial Lock-In: The policy becomes anchored to specific spatial targets (e.g., "left cup") and fails to respond to alternative spatial cues in the prompt (e.g., "right cup"), even if the underlying manipulation skill is learned.

Existing remedies typically rely on massive data augmentation or external supervision (e.g., foundation models), which increases training complexity and deployment costs.

2. Methodology: The DeLock Framework

The authors propose DeLock, a framework that avoids external supervision by preserving and then exploiting the model's internal pre-trained knowledge. It consists of two tightly coupled components:

A. Visual Encoder Weight-Drift Regularization (Training Time)
The authors identify that instruction grounding is mediated by visual features. In standard fine-tuning, the visual encoder often "drifts" too far toward the narrow post-training distribution, causing representation collapse.

• Mechanism: They augment the standard Behavioral Cloning (BC) loss with an $L_2$ regularization term that penalizes the distance between the post-trained visual encoder parameters ($\theta_v$) and the original pre-trained parameters ($\theta_{pre}^v$):
  $$\mathcal{L}_{DeLock} = \mathcal{L}_{BC} + \lambda \|\theta_v - \theta_{pre}^v\|_2^2$$
• Goal: This preserves the pre-trained visual grounding (the ability to "see" and "distinguish" various concepts) while allowing the language and action modules to adapt to the new task via LoRA.

B. Contrastive Prompt Guidance (CPG) (Inference Time)
To overcome residual biases during execution, DeLock uses a test-time steering mechanism based on the model's own denoising dynamics.

• Mechanism: During the action generation (flow-matching) process, the model performs two forward passes at each denoising step: one with the novel prompt ($\tau^+$) and one with the trained (negative) prompt ($\tau^-$).
• Steering Rule: The denoising vector field is updated by contrasting these two flows:
  $$v_t^{CPG} = v_t^{\tau^-} + w(v_t^{\tau^+} - v_t^{\tau^-})$$
• Goal: This amplifies the direction toward the novel instruction while suppressing the bias toward the training data, effectively "steering" the robot toward the new target.

3. Key Contributions

1. Formalization of Lock-In: Defined and distinguished Concept and Spatial Lock-In as fundamental failure modes in low-data VLA adaptation.
2. DeLock Framework: Introduced a dual-stage approach (Weight-Drift Regularization + Contrastive Prompt Guidance) that requires no external models or additional data.
3. New Benchmark: Developed a dedicated evaluation suite comprising 8 tasks (4 simulation via LIBERO, 4 real-world via DROID) specifically designed to probe concept and spatial reasoning.

4. Results

• Superior Generalization: Across eight evaluations, DeLock consistently outperformed strong baselines like RETAIN (weight interpolation).
• Efficiency vs. Scale: DeLock, trained on very limited data, matched or exceeded the performance of state-of-the-art generalist policies (like $\pi_{0.5}$-DROID) that were post-trained on significantly larger, curated datasets.
• Ablation Insights:
  • Without Visual Regularization, the model fails to ground novel concepts.
  • Without CPG, the model retains concept grounding but fails to overcome spatial biases during rollout.
• Mechanistic Success: Qualitative analysis (cross-attention heatmaps and counterfactual rollouts) confirmed that DeLock successfully reallocates attention to novel objects and redirects denoising trajectories toward novel spatial targets.

5. Significance

This work is significant because it addresses a critical bottleneck in deploying foundation models for robotics: the cost of data. By proving that a VLA's internal pre-trained priors are sufficient to maintain steerability, DeLock provides a path toward highly adaptable, task-specific robots that can be trained on minimal demonstrations without losing the "intelligence" and "generalization" inherent in their large-scale pre-training.