Technical Summary: RePO-VLA

Problem Statement

Vision-Language-Action (VLA) models have advanced general-purpose robot control but remain brittle in long-horizon, contact-rich manipulation tasks. Current approaches rely heavily on supervised fine-tuning (SFT) over successful demonstrations, creating a "failure-utilization blind spot." When execution drift occurs—due to grasp pose deviations, object slippage, or timing errors—standard VLAs lack the supervision to recover. Furthermore, existing methods often discard failed rollouts entirely, ignoring that their early phases may contain valid approach behaviors. Conversely, naively imitating entire recovery trajectories (which include both failure history and corrective actions) leads to causal confusion, where the policy learns to associate the preceding error with the recovery action, rather than learning to react to the current adverse state.

Methodology: RePO-VLA

RePO-VLA (Recovery-Driven Policy Optimization for Vision-Language-Action Model) is a framework designed to learn from successful, failed, and recovered trajectories under a unified progress signal. It operates in two distinct phases:

1. Recovery-Aware Initialization (RAI)

The first phase addresses the "causal confusion" problem where a policy might learn that a specific failure history is a necessary trigger for recovery.

• Trajectory Slicing with History Reset (TSHR): Raw recovery trajectories are sliced to remove the failure prefix. The observation history buffer is reset at the start of the corrective segment. This forces the policy to learn recovery as a state-conditioned skill based on the current adverse state (e.g., an empty gripper or misaligned object) rather than a replay of the specific mistake that caused it.
• Training: The policy is fine-tuned on a mix of expert demonstrations and these reset recovery segments ($D_{expert} \cup D_{reset}^{rec}$), balancing nominal imitation with recovery learning.

2. Value-Conditioned Refinement (VCR)

The second phase restores the full rolling history (including failure prefixes) and introduces a dense semantic value signal to distinguish between useful, drifting, and corrective behaviors.

• Progress-Aware Semantic Value Function (PAS-VF): A lightweight adapter network aligns spatiotemporal video features with language instructions in a shared semantic manifold. It learns to estimate task progress by measuring the cosine similarity between the current trajectory prefix and a cluster of successful reference trajectories.
• Dense Labeling:
  • Success/Recovery: Frames maintaining or restoring progress are labeled $v_t = 1.0$.
  • Error Segments: Prefixes leading to adverse states are labeled $v_t = 0.0$ to prevent averaging with corrective actions.
  • Pure Failures: Early kinematic priors are preserved via a reliability decay mechanism ($v_t = V(\tau_{fail}) \cdot (1 - t/T)^\alpha$), while terminal breakdowns are down-weighted.
• Value-Conditioned Training: The policy is augmented with a value token ($e_{val}$) derived from the dense labels. During training, the policy learns to map the same adverse observation and history to different action modes depending on the desired value.
• Deployment: At inference, the policy uses the standard rolling history buffer (without resetting) but fixes the value token to $v = 1.0$. This biases the action distribution toward the learned "success manifold," enabling autonomous recovery without online failure detectors or heuristic retry rules.

Key Contributions

1. RePO-VLA Framework: A two-phase (RAI + VCR) architecture that structurally reuses failed rollouts. It eliminates causal confusion via history resetting and enables value-differentiated actions via dense progress labels.
2. FRBench: A new benchmark for recovery evaluation featuring standardized error injection (E1–E4: premature close, grasp slip, position offset, orientation mismatch) and a phase-based protocol that isolates nominal proficiency from corrective capability in both simulation (RoboTwin) and real-world bimanual tasks.
3. Empirical Validation: Demonstrated that explicit recovery supervision and dense value guidance significantly improve robustness. The paper shows a positive scaling trend where increasing the density of recovery data further improves performance.

Experimental Results

• Simulation (FRBench-Sim): Across 46 bimanual tasks, RePO-VLA significantly outperformed baselines (including $\pi_0$, $\pi_0.5$, and RDT) under injected failures. In randomized settings with adversarial perturbations, RePO-VLA achieved an average success rate of 43.0%, compared to 15.4% for the $\pi_0.5$ baseline.
• Real-World Evaluation: Tested on Dobot X-Trainer arms for tasks like pouring water, cooking, and towel folding.
  • Under adversarial perturbations, the baseline $\pi_0.5$ achieved 20.0% success, while RePO-VLA (Phase I only) reached 37.5%.
  • Data Scaling: Increasing recovery data density from 1x to 4x further improved performance. The Full (4x) variant achieved 75% success under adversarial conditions, surpassing the Phase I baseline and demonstrating that denser coverage of failure-recovery neighborhoods is critical.
• Ablation Studies: Confirmed that history resetting (TSHR) is essential to prevent causal confusion and that value conditioning ($v=1.0$) provides a stable control signal superior to heuristic retries.

Significance and Claims

The paper claims that RePO-VLA addresses the brittleness of current VLA models by fundamentally changing how failure data is utilized. Rather than discarding failures or treating recovery episodes as uniformly positive, the framework decomposes them into distinct semantic roles:

• Useful prefixes of failures are retained as kinematic priors.
• Terminal breakdowns are safely down-weighted.
• Corrective actions are learned as state-conditioned skills, decoupled from the specific failure history that preceded them.

The authors posit that this approach allows a single policy to handle both nominal execution and autonomous recovery without switching modules or relying on external failure detectors. The results suggest that for reliable long-horizon manipulation, especially in contact-rich environments, the ability to recover from drift is as critical as the ability to execute the nominal plan, and this capability can be learned directly from mixed-quality interaction data.