Technical Summary: µVLA – On Recurrent Memory for Partially Observable Manipulation in VLA Models

Problem Statement

Vision-Language-Action (VLA) models typically formulate robot control as next-token prediction over multimodal sequences, assuming the current observation (potentially augmented by a short context window) is a sufficient statistic for control. This Markovian assumption holds in fully observable settings but fails under partial observability, where decision-relevant information is transient, occluded, or established earlier in an episode.

Existing approaches to memory in VLAs often combine recurrence with other complex machinery, such as external retrieval systems, hierarchical memory structures, auxiliary objectives (e.g., predicting past observations), or task-specific architectural changes. Consequently, the specific contribution of recurrence itself remains entangled with these surrounding mechanisms, making it difficult to isolate whether performance gains stem from the recurrent state or the auxiliary components.

Methodology: µVLA

The authors present µVLA, a controlled isolation study of recurrence within a strong, pretrained VLA backbone (OpenVLA-OFT). The core formulation augments the transformer with a minimal set of learnable memory tokens ($M_t \in \mathbb{R}^{m \times d}$) carried across timesteps and updated via standard self-attention.

Key Design Choices

1. Minimalist Architecture: The model adds only $m$ learnable memory tokens to the input sequence (placed between proprioception and text tokens). No new layers, external memory banks, or retrieval modules are introduced.
2. Training Regime:
   • Episodic Dataloader: Unlike standard shuffled training, the model is trained on temporally ordered episodes using a round-robin dataloader to maintain state continuity.
   • Truncated Backpropagation Through Time (TBPTT): The model is fine-tuned end-to-end with TBPTT over a window of $K$ steps. Gradients flow through the recurrent chain within the window but are detached at the boundary.
   • No Auxiliary Losses: The model is trained solely on the standard action prediction loss (L1 regression), without auxiliary objectives like past-observation prediction.
3. Attention Mask Guard: A critical design element is a modified attention mask that prevents memory tokens from attending to action tokens. This prevents a degenerate solution where the model trivially copies the predicted action chunk into memory, forcing it to encode task-relevant environmental cues instead.
4. Receding-Horizon Inference: To align training and inference, the model uses receding-horizon control, querying the model and executing only the first action of a predicted chunk at every environment step. This ensures memory updates occur at the correct cadence, avoiding the train-test mismatch inherent in open-loop chunking.

Experimental Setup

The study evaluates µVLA on two benchmarks:

• MIKASA-Robo: A partially observable manipulation benchmark designed to isolate memory dependence (e.g., cue-recall, occlusion tracking, sequential memory).
• LIBERO: A fully observable benchmark used as a control to test for performance regression in Markovian settings.

The authors vary three parameters to create a family of models: memory width ($m$), TBPTT length ($K$), and the memory update rule (TBPTT vs. detached Exponential Moving Average).

Key Contributions

1. Controlled Isolation Study: This work provides the first controlled isolation of recurrence in a pretrained VLA backbone, varying recurrence as a single experimental axis without entangled retrieval, compression, or auxiliary losses.
2. µVLA Family: The introduction of a parameterized family of recurrent fine-tunes of OpenVLA-OFT that share the same backbone, optimizer, and inference protocol, differing only in their recurrent state configuration.
3. Capability Calibration: The paper calibrates the "capability envelope" of minimal in-backbone recurrence, identifying specific regimes where it is sufficient and where it fails.

Results

Performance on Partially Observable Tasks (MIKASA-Robo)

• Training Tasks: On the five training tasks (covering cue-recall, occlusion, and sequential memory), the best µVLA configuration ($m=64, K=2$) improved the average success rate (SR) from 0.42 (memoryless baseline) to 0.84.
  • Significant gains were observed in cue-recall tasks (e.g., RememberColor5: 0.04 $\to$ 0.93).
• Held-Out Tasks (Matched Semantics): On tasks sharing the same memory semantics as training but with different difficulty levels, the best µVLA achieved 0.23 SR, compared to 0.07 for the memoryless baseline.
• Held-Out Tasks (Novel Semantics): On tasks requiring memory structures not present in the training mixture (e.g., shape-only memory when trained on color), performance remained near the baseline, indicating that minimal recurrence does not generalize to entirely new memory semantics without retraining.
• Truncation Length ($K$): A U-shaped performance curve was observed. $K=2$ outperformed both $K=1$ (no cross-step gradients) and $K=8$ (longer gradients), suggesting a sweet spot for credit assignment versus signal resolution.

Performance on Fully Observable Tasks (LIBERO)

• No Regression: On the fully observable LIBERO suite, the strongest recurrent variant ($m=64, K=8$) achieved 96.2% average SR, matching or exceeding the memoryless OpenVLA-OFT baseline. This confirms that adding a recurrent channel does not degrade performance in Markovian settings.
• Inference Sensitivity: The results highlighted the importance of receding-horizon inference. When the same µVLA model was evaluated with open-loop chunking (updating memory only once per chunk), performance collapsed (e.g., Long-10 SR dropped from 95.8% to 5.4%), demonstrating the critical alignment between training and inference cadence.

Diagnostics and Interventions

• Causal Interventions: Replacing memory tokens with Gaussian noise during inference caused a sharp drop in success rates on memory-dependent tasks (e.g., RememberColor5: 0.94 $\to$ 0.09), confirming the policy actively reads the recurrent state.
• Attention Rollouts: Attention analysis showed memory tokens concentrating on cue regions (e.g., a lamp or a grid) during the observation phase and diffusing during occlusion, aligning with phase transitions in the task.
• Generalization: The model showed partial robustness to out-of-distribution (OOD) cue identities (e.g., color swaps) but degraded significantly when the phase length (delay between cue and action) exceeded the training horizon.

Significance and Claims

The authors position µVLA not as a new foundation model architecture, but as a calibration of the capability envelope of minimal recurrence.

• Recurrence is a Strong Ingredient: The results argue that recurrence itself is a powerful, understudied component in VLA systems. Even a small set of learnable tokens, trained end-to-end without auxiliary losses, can qualitatively change the operating regime of a VLA, enabling it to solve partially observable tasks that are impossible for memoryless policies.
• Limits of Minimal Recurrence: The study reveals clear limits. Minimal recurrence is sufficient for tasks where the memory requirement is represented in the training distribution and the temporal horizon is short. However, it does not automatically generalize to novel memory semantics or very long-horizon dependencies without additional structured memory mechanisms.
• Design Implications: The work suggests that future memory-augmented VLAs should carefully disentangle the contributions of recurrence from retrieval and hierarchical structures. The "sweet spot" for TBPTT length ($K=2$) and the necessity of the attention guard are specific, actionable findings for designing efficient recurrent VLAs.

In summary, the paper demonstrates that recurrence alone, when isolated and properly trained, is a potent mechanism for handling partial observability in robotic manipulation, provided the task semantics are within the model's learned distribution and the inference protocol aligns with the training cadence.