Technical Summary: Adaptive Failure-Informed Learning (AFIL) for Vision-Language-Action Models

1. Problem Statement

Vision-Language-Action (VLA) models have emerged as a scalable paradigm for robotic manipulation, leveraging large-scale pretrained vision-language representations to enable embodied decision-making. However, current VLA models suffer from significant brittleness during deployment. This limitation stems primarily from their reliance on success-only behavioral cloning. Training data typically consists of teleoperated trajectories collected in confined environments, which lack corrective signals for suboptimal or unseen states.

Consequently, policies trained solely on successful demonstrations tend to overfit to narrow behavioral manifolds. When minor execution errors occur, they rapidly compound into unrecoverable, out-of-distribution failures. While recent work suggests learning from failure data can improve robustness, existing approaches face critical limitations:

• Manual Design: Many methods rely on handcrafted failure taxonomies (e.g., defining specific categories like "slip" or "incomplete grasp"), which require domain expertise and fail to capture the stochastic diversity of real-world failures.
• Human Dependency: Recovery often depends on task-specific heuristics or real-time human intervention, limiting scalability.
• Computational Cost: Applying negative guidance (similar to negative prompting in image generation) to sequential robotic actions is non-trivial. Conventional methods often require separate models for desired and undesired distributions, which is computationally prohibitive for VLA systems built on large Vision-Language Model (VLM) backbones.

2. Methodology: Adaptive Failure-Informed Learning (AFIL)

The authors propose Adaptive Failure-Informed Learning (AFIL), an end-to-end framework designed for diffusion- and flow-matching-based VLA policies. AFIL leverages failure trajectories as adaptive negative guidance to steer action generation away from failure-prone regions.

2.1 Dual Action Generator (DAG) Architecture

AFIL introduces a Dual Action Generator (DAG) architecture that jointly learns from both success and failure data while maintaining parameter efficiency:

• Shared Backbone: Both generators share a common pretrained VLM backbone ($\Phi_{VLM}$) for visual perception and semantic grounding.
• Dual Heads: The architecture employs two distinct action generators:
  1. Success Generator ($\mathcal{G}_{succ}$): Trained exclusively on successful trajectories to model the distribution $p_{succ}(a | o, \ell)$.
  2. Failure Generator ($\mathcal{G}_{fail}$): Trained exclusively on failure trajectories to model the distribution $p_{fail}(a | o, \ell)$.
• Online Failure Generation: Instead of relying on manual annotation, AFIL generates failure trajectories online by rolling out a pretrained VLA under task-conditioned execution. These trajectories capture emergent and compounding error modes naturally.

2.2 Training Strategy

The model is trained on a mixed dataset $\mathcal{D} = \{\mathcal{D}_s, \mathcal{D}_f\}$.

• For diffusion-based policies, each generator predicts noise $\epsilon_\theta$ and minimizes the standard diffusion objective on its respective dataset.
• For flow-matching policies, the heads predict velocity fields to transport samples from noise to the data distribution.
• Crucially, the underlying generative training objectives remain unchanged; the innovation lies in the joint training structure and the inference-time guidance mechanism.

2.3 Adaptive Failure-Informed Sampling

At inference, AFIL employs a dynamic guidance mechanism inspired by Negative Prompting in text-to-image generation. Rather than using a static guidance scale, AFIL adaptively modulates the strength of the negative guidance based on the divergence between the success and failure predictions at each diffusion step.

The guided score $\epsilon^*_{FI}$ is computed as:
$$\epsilon^*_{FI} = \epsilon_{succ} - \hat{\lambda}_\eta(\mathbf{a}^\eta) (\epsilon_{fail} - \epsilon_{succ})$$
Where the adaptive guidance scale $\hat{\lambda}_\eta$ is determined by the cosine similarity between the predicted noise vectors of the success and failure generators:
$$\hat{\lambda}_\eta(\mathbf{a}^\eta) = \alpha (1 - \cos(\epsilon_{succ}, \epsilon_{fail}))$$

Rationale for Adaptivity:

• Early Stages: At the beginning of an episode, successful and failed rollouts often share similar reaching motions. Here, the cosine similarity is high, resulting in a low guidance scale to avoid suppressing valid actions.
• Divergence Stages: As the trajectory progresses, if a failure mode emerges (e.g., dropping an object), the predictions diverge. The cosine similarity decreases, increasing the guidance scale to actively repel the policy from the failure manifold and steer it toward corrective behaviors.

3. Key Contributions

The paper outlines three primary contributions:

1. AFIL Framework: The introduction of Adaptive Failure-Informed Learning for diffusion- and flow-based VLAs, utilizing online-generated failure rollouts as a principled form of adaptive negative guidance for sequential action generation.
2. DAG-VLA Architecture: A novel architecture that jointly learns success and failure behaviors with a shared VLM backbone, enabling efficient failure-aware policy learning with limited parameter overhead.
3. Empirical Validation: Demonstration that AFIL consistently improves success rates and robustness across diverse in-domain and out-of-domain manipulation tasks, including both short- and long-horizon settings.

4. Experimental Results

The authors evaluated AFIL on the ManiSkill and LIBERO benchmarks using a $\pi0.5$ backbone.

4.1 In-Domain Performance (ManiSkill)

• Short-Horizon Tasks: AFIL improved success rates significantly over success-only baselines. For example, in "Lift the peg upright," success rates increased from 70.7% (Success Only) to 90.7% (Adaptive-FIL).
• Long-Horizon Tasks: The gains were most pronounced in complex, multi-step tasks where errors compound. In "Place can on the plate and banana on the plate," success rates rose from 39.3% to 71.3%, outperforming static failure-informed sampling by over 12%.
• Ablation: The study confirmed that adaptive guidance ($\alpha=1.0$) outperforms static guidance, particularly in balancing the repulsion of failure modes without suppressing valid actions during early trajectory stages.

4.2 Benchmark Evaluation (LIBERO)

• AFIL achieved state-of-the-art performance across all four LIBERO suites (Spatial, Object, Goal, and Long).
• On the challenging LIBERO-10 (long-horizon) suite, the method improved the baseline $\pi0.5$ performance from 92.4% to 95.8%.
• The average success rate across all suites improved from 96.9% to 98.4%, demonstrating that even strong pretrained policies benefit from adaptive failure guidance.

4.3 Out-of-Domain (OOD) Evaluation

• In scenarios involving unseen objects, novel geometries, and cluttered environments, the success-only policy degraded significantly (e.g., 10.7% on Fruit Sorting).
• AFIL demonstrated robust generalization, achieving 64.0% success on Fruit Sorting, significantly outperforming both the success-only baseline and the "Success + Failure Correction" baseline without adaptive guidance.

5. Significance and Claims

The paper claims that AFIL offers a scalable and efficient mechanism for robust robotic action generation without requiring:

• Handcrafted failure modes or taxonomies.
• Task-specific recovery heuristics.
• Separate, large-scale VLA models for failure detection.

By explicitly modeling failure distributions and using them as adaptive negative guidance, AFIL addresses the brittleness inherent in success-only training. The authors posit that this approach provides a promising direction for developing VLA policies capable of generalizing to realistic deployment settings where errors, distribution shifts, and the need for recovery are unavoidable. The method is presented as model-agnostic, applicable to any diffusion- or flow-matching-based VLA policy.