HELM: Harness-Enhanced Long-horizon Memory for Vision-Language-Action Manipulation

1. Problem Statement

Vision-Language-Action (VLA) models (e.g., OpenVLA, Octo) have demonstrated strong performance on short-horizon robotic manipulation tasks. However, they fail systematically on long-horizon tasks (tasks requiring 5+ sequential subgoals), even when the context window is extended.

The authors identify that this failure is not merely a result of insufficient context length but stems from three structural deficiencies in the standard reactive execution loop:

1. Memory Gap (FM): Fixed context windows discard evidence of completed subgoals. The model "forgets" past states, leading to redundant actions (e.g., trying to place an object that was already placed) and corrupted task states.
2. Verification Gap (FV): Actions are proposed reactively without pre-execution validation. Infeasible grasps, collisions, or semantically incorrect actions are executed silently, propagating errors.
3. Recovery Gap (FR): When an action fails, the model continues executing subsequent subgoals based on the corrupted state, causing cascading failures rather than attempting to recover or rollback.

The paper argues that simply increasing the context window (e.g., from $H=8$ to $H=32$) yields negligible improvements (only +5.4% Task Success Rate), confirming the need for a structural architectural change.

2. Methodology: The HELM Framework

HELM is a model-agnostic framework that wraps a frozen VLA backbone with three dedicated components to address the three gaps. It operates in a "Retrieve $\to$ Propose $\to$ Verify $\to$ Execute/Recover" loop.

A. Episodic Memory Module (EMM)

• Function: Addresses the Memory Gap.
• Mechanism: Maintains a key-value store of "keyframes."
  • Write Policy: Stores frames upon subgoal completion, detected failure, or periodic checkpoints ($\Delta c = 20$ steps).
  • Retrieval: Uses CLIP (ViT-B/32) to encode the current observation and retrieves the top-$k$ most semantically similar historical keyframes.
  • Integration: The retrieved keyframes are serialized into structured text and injected into the VLA's language input, providing the model with long-term context beyond its fixed window.

B. State Verifier (SV)

• Function: Addresses the Verification Gap. This is the core learning contribution.
• Mechanism: A lightweight, learned 3-layer MLP trained to predict the probability of action failure ($p_{fail}$) before execution.
• Input: It takes a memory-augmented context vector:
  $$\hat{o}_t = [\phi(o_t); k_{top-1}]$$
  concatenated with the projected action ($a_t$) and subgoal embedding ($g_t$).
• Training: Trained on 50k rollout steps from the VLA. The label is positive if a failure occurs within the next 5 steps.
• Significance: Unlike rule-based checkers (which miss semantic errors) or uncertainty ensembles (which are computationally expensive), the SV learns to reason jointly over visual state, proposed action, and task history.

C. Harness Controller (HC)

• Function: Addresses the Recovery Gap.
• Mechanism: Orchestrates the execution loop.
  • If $p_{fail} > \theta_v$ (threshold), the HC triggers a recovery sequence.
  • Rollback: Retrieves the most recent successful checkpoint from the EMM, resets the environment state (or prompts the VLA to return to that state), and re-pushes the failed subgoal.
  • Forward Recovery: A variant (HELM-Fwd) generates a plan to recover from the current corrupted state if rollback is physically impossible.

3. Key Contributions

1. Failure Taxonomy & Analysis: Empirical evidence showing that long-horizon VLA failures are structural (Memory, Verification, Recovery gaps) and cannot be solved by scaling context length alone.
2. HELM Framework: A novel architecture integrating EMM, SV, and HC.
3. Learned State Verifier (SV): The first learned pre-execution failure predictor conditioned on episodic memory. It outperforms rule-based verifiers and ensemble uncertainty baselines while being computationally efficient.
4. LIBERO-Recovery Protocol: A new evaluation benchmark injecting controlled perturbations (object displacement, gripper state flips) to specifically test failure recovery capabilities.

4. Experimental Results

Experiments were conducted on LIBERO-LONG (10 tasks, avg. 5.8 subgoals) and CALVIN.

• Performance Gains:

  • OpenVLA (Baseline): 58.4% Task Success Rate (TSR).
  • OpenVLA + Extended Context ($H=32$): 63.8% TSR (+5.4 pp).
  • HELM (OpenVLA): 81.5% TSR (+23.1 pp over baseline).
  • HELM (Octo): 72.8% TSR (proving model-agnosticism).

• Ablation Studies:

  • Removing EMM drops TSR by 11.2%.
  • Removing SV drops TSR by 8.4%.
  • Crucially, an SV trained without memory context only improves TSR by 2.3%, confirming the EMM-SV coupling is essential.
  • Rollback contributes significantly to the Recovery Success Rate (RSR), improving it by 31.1% compared to forward-only recovery.

• Efficiency: The SV adds only ~12ms per step (15% overhead) and is significantly cheaper than 5x ensemble uncertainty baselines.

5. Significance and Conclusion

HELM demonstrates that execution-time augmentation is a more effective strategy for long-horizon manipulation than simply scaling backbone models or context windows. By explicitly separating memory retrieval, feasibility verification, and recovery logic, HELM bridges the gap between short-horizon VLA capabilities and the requirements of complex, multi-step robotic tasks.

The work highlights that the "memory gap" is not just about storing more pixels, but about retrieving relevant semantic context to inform both the policy and a learned verifier. The proposed LIBERO-Recovery protocol provides a standardized way to evaluate robustness against real-world disturbances, a critical step toward deploying robots in unstructured environments.

Limitations: The SV requires environment-specific training data (~50k steps), and rollback assumes reversible actions (though HELM-Fwd mitigates this). Sim-to-real transfer remains a future challenge due to domain shifts in visual embeddings and sensor noise.