🤖 machine learning

REFACTOR-VLA: Unsupervised Library Learning of Typed Motor Programs

REFACTOR-VLA is a wake/sleep framework that learns reusable, typed motor programs by clustering action fragments via a behavioral-equivalence kernel and a latent world model, achieving state-of-the-art performance on long-horizon LIBERO tasks through a library-conditioned decoder and a training objective that prioritizes clustering quality over model capacity.

Original authors: Riyaaz Shaik, Chandru Venkataraman

Published 2026-09-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Riyaaz Shaik, Chandru Venkataraman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots are getting better at seeing the world and moving through it, but they still struggle with the kind of complex, multi-step jobs that humans handle with ease. When a person makes a sandwich, they do not think about every single muscle twitch required to grab the bread, spread the butter, or slice the tomato. Instead, they rely on a mental library of familiar actions—grasping, spreading, cutting—that they can string together in different orders. Current artificial intelligence models for robots often lack this ability to organize behavior. They tend to generate raw, moment-to-moment commands without grouping them into reusable, well-defined skills. This makes it difficult for them to plan ahead for long tasks or to explain what they have learned. Researchers are now trying to teach machines to build their own internal libraries of skills, allowing them to break down complicated jobs into smaller, manageable pieces that can be reused and combined.

A team of researchers at Apple has developed a new system called REFACTOR-VLA that attempts to solve this problem by teaching a robot to discover its own skills without human labels. The system works in two alternating phases, much like how a human might practice a new movement and then rest to let the brain organize the memory. In the first phase, the robot learns to predict what will happen next if it performs a certain sequence of movements. It builds a mental model of the world, understanding how its actions change the environment. In the second phase, the system looks at the vast amount of movement data it has collected and tries to find patterns. It asks a simple but difficult question: do two different sequences of movements produce the same result? If a robot moves its arm in a circle to pick up a cup, or moves it in a straight line to do the same thing, the system needs to recognize that both paths achieve the same goal.

To answer this, the researchers created a special tool that measures the outcome of actions rather than just their appearance. Instead of looking at the raw video of the robot moving, the system simulates the action in its learned mental model to see where the robot ends up and what reward it gets. If two different movement paths lead to the same final state and the same reward, the system treats them as equivalent. It then groups these similar paths together into a single, reusable skill. Once a group is formed, the system tries to write a simple, structured program that describes the common pattern of that skill. This program is then added to a library. The robot can later use this library to plan new tasks, calling upon these pre-packaged skills instead of calculating every tiny movement from scratch.

The researchers tested this approach on a standard set of robot tasks involving moving objects in a simulated environment. They discovered something surprising about how these systems learn. A common belief in artificial intelligence is that making a model larger and more complex always leads to better performance. However, when the researchers increased the size of their world model from roughly 188 million parameters to 430 million, the system actually performed worse on every single test suite. The larger model failed to organize the skills correctly, suggesting that simply adding more computing power is not the solution.

Instead, the key to success lay in how the model was trained. The researchers found that adding a specific type of learning objective during the initial training phase dramatically improved the results. This objective helped the system distinguish between different tasks more clearly, allowing it to group similar movements together much more effectively. With this change, the system outperformed the strongest existing methods on all four major test suites, improving its average score by a significant margin. The system successfully built a library of three distinct, reusable skills for a specific task, and a robot using this library was able to rewrite every single demonstration it had seen using these new skills.

The study also revealed that the system's ability to find these skills depends heavily on the type of data it analyzes. While the system successfully created a library of language-based instructions, such as "pick up the object and place it in the basket," it failed to find reusable patterns in the raw motor commands themselves. This suggests that the system is better at understanding the high-level goals of a task than the low-level physical details of how to move. The researchers measured the consistency of their results across many different training runs and found that the system reliably discovered similar skill groupings, with a high degree of agreement between different versions of the model.

This work demonstrates that the way a robot learns to organize its experiences is more important than the sheer size of its brain. By focusing on the outcomes of actions and using a structured method to group them, the system can build a library of skills that helps it tackle complex, long-term tasks. The findings challenge the idea that bigger is always better and point toward a future where robots can learn to think in terms of reusable actions, just as humans do. The researchers have made their code and data available, allowing others to verify these results and build upon this new approach to robotic learning.

Technical Summary: REFACTOR-VLA

Problem Statement

Current Vision-Language-Action (VLA) models (e.g., OpenVLA, $\pi_0$ , RT-2) are largely "monolithic," generating raw motor commands or short action sequences without organizing behaviors into reusable, well-defined abstractions. This limitation results in poor performance on long-horizon tasks and a lack of interpretability regarding learned behaviors.

Existing skill-discovery approaches struggle with the core problem of defining behavioral equivalence in continuous action spaces. Methods like AtomicVLA and AtomSkill rely on clustering contrastive embeddings, while BLADE and LRLL use Large Language Models (LLMs) to judge equivalence. However, these approaches are often uncalibrated to the robot's specific dynamics: they may treat trajectory fragments with identical physical outcomes but different action paths as distinct, or group fragments with similar surface forms but different dynamical consequences. Furthermore, simply scaling world model capacity has not been proven to consistently improve skill representations.

Methodology: REFACTOR-VLA

REFACTOR-VLA is a unified framework that combines the systems-level strengths of VLA models with the formal guarantees of wake/sleep library learning. It operates on a three-phase schedule to learn reusable, typed motor programs.

1. Core Architecture

The system consists of four modules:

Latent World Model ( $M_\phi$ ): A DreamerV3-style hierarchical model with a frozen DINOv2 visual encoder. It learns to predict future states and returns in a latent space.
Behavioral-Equivalence Kernel (BEK): A divergence metric $D_\phi(\tau, \tau')$ $D_{ϕ} (τ, τ^{'})$ that quantifies the difference between trajectory fragments. It is defined as a weighted sum of:
- Value Difference: The absolute difference in expected returns when fragments are rolled out from the same initial state.
- Wasserstein Distance ( $W_2$ ): The distance between $k$ -step latent rollout distributions.
- This kernel is implemented via a Siamese amortizer ( $k_\chi$ ), a lightweight transformer distilled from $M_\phi$ to efficiently compute embeddings for clustering.
Typed Program Emitter (TPE): A causal-decoder Transformer that generates typed $\lambda$ -terms (programs) over a Hindley–Milner vocabulary (including primitives like Twist, Wrench, GripperPhase, Pose, and Lang).
Library-Conditioned Action Decoder (LCAD): A rectified-flow decoder that produces 16-step action chunks, conditioned on the active library of abstractions.

2. The Wake/Sleep Loop

The training alternates between two phases:

Phase A (World-Model Warmup): $M_\phi$ is pre-trained on raw demonstrations. Crucially, the authors introduce an auxiliary supervised contrastive InfoNCE loss during this phase to encourage task-discriminative directions in the latent space.
Phase B (Wake Phase): The policy ( $\pi_\theta$ ) is optimized using the current library of skills. The TPE generates typed programs, and the LCAD produces actions.
Phase C (Sleep Phase): The system discovers new skills.
1. Clustering: Trajectory fragments are clustered using the BEK ( $D_\phi$ ) and KMeans.
2. Abstraction: Top-down syntactic anti-unification extracts common grammatical sub-structures from the clustered fragments, compiling them into typed $\lambda$ -programs.
3. Admission Gates: A candidate abstraction is admitted to the library only if it satisfies:
  - BEK Soundness: The abstraction must be behaviorally equivalent to the original fragments within a threshold $\epsilon$ .
  - Return Preservation: The abstraction must not degrade the expected return (verified via rollouts).
  - Minimum Description Length (MDL): The abstraction must provide a net gain in data compression (> 4 nats).

Key Contributions

BEK Formulation: A fragment-level divergence metric based on value differences and Wasserstein distances of latent rollouts, used as a clustering kernel that respects system dynamics.
Typed-Lambda Emitter & LCAD: A system that generates type-checked lambda programs over a structured vocabulary and conditions a rectified-flow action decoder on these programs.
Wake/Sleep Loop with MDL-Gated Admission: A rigorous framework where abstractions are only admitted if they satisfy joint soundness, return-preservation, and compression criteria.
Empirical Validation: Comprehensive evaluation on the full LIBERO benchmark suite, including ablation studies and cross-provider convergence analysis.

Results

The paper presents two primary findings that challenge conventional scaling laws:

Capacity Falsification: Increasing the latent world model size from 188M to 430M parameters worsened performance on all 4 LIBERO suites (decreasing NMI by 0.018 to 0.146). This disproves the hypothesis that larger world models inherently yield better skill representations.
Objective Shape Dominance: Adding an auxiliary supervised InfoNCE loss during the Phase A warmup significantly improved skill discovery. This change increased the Normalized Mutual Information (NMI) of the BEK clustering by a mean of +0.252 (relative improvement of 50.7%) across all suites.
- Head-to-Head Performance: With the InfoNCE objective, REFACTOR-VLA outperformed the strongest published baselines (AtomicVLA, AtomSkill, BLADE, LRLL) on all 4 LIBERO suites, achieving a mean improvement of $\Delta = +0.184$ .
- Cross-Provider Convergence: In a cross-provider analysis ( $n=12$ ), the mean pairwise NMI was 0.705 with a 95% bootstrap confidence interval of [0.683, 0.729]. While the mean exceeds the 0.70 threshold, the lower bound is slightly below it, indicating the result is "threshold-borderline" but robustly above the 0.60 substantial-effect floor.
- Library Learning: The system successfully discovered the first structured task-language library for LIBERO (3 abstractions on the Lang slot) and successfully rewrote 100% of sampled demonstrations using these skills.

Significance and Claims

The paper claims that the shape of the training objective is a more critical lever for skill discovery than model capacity. Specifically, the introduction of a supervised contrastive objective (InfoNCE) during world-model warmup aligns the latent space with task-discriminative bisimulation classes, enabling effective behavioral equivalence clustering.

The authors position REFACTOR-VLA as a step toward interpretable, compositional robot learning. By framing skill discovery as a density-estimation problem with formal guarantees (MDL and return preservation), the system avoids the "black box" nature of monolithic VLAs. The results suggest that for long-horizon robotic tasks, optimizing the structure of the learning signal is more effective than simply scaling parameters.

The paper remains modest regarding open-corpus applications, noting that the current InfoNCE approach relies on task labels available in the LIBERO dataset, and future work is needed to adapt these mechanisms to open-vocabulary, label-free settings (e.g., using VLM-generated pseudo-labels or self-supervised contrastive sampling).