🤖 machine learning

Habilis- $β$ : A Fast-Motion and Long-Lasting On-Device Vision-Language-Action Model

The paper introduces Habilis- $\beta$ , a fast-motion and long-lasting on-device Vision-Language-Action model that achieves superior continuous-run performance and state-of-the-art results on the RoboTwin 2.0 leaderboard by leveraging the Productivity-Reliability Plane for evaluation and integrating language-free pre-training, cyclic task post-training, and advanced motion shaping techniques.

Original authors: Tommoro Robotics, :, Jesoon Kang, Taegeon Park, Jisu An, Soo Min Kimm, Jaejoon Kim, Jinu Pahk, Byungju Kim, Junseok Lee, Namheon Baek, Sungwan Ha, Hojun Baek, Eduardo Ayerve Cruz, Wontae Kim, Junghyeo

Published 2026-02-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tommoro Robotics, :, Jesoon Kang, Taegeon Park, Jisu An, Soo Min Kimm, Jaejoon Kim, Jinu Pahk, Byungju Kim, Junseok Lee, Namheon Baek, Sungwan Ha, Hojun Baek, Eduardo Ayerve Cruz, Wontae Kim, Junghyeon Choi, Yousuk Lee, Joonmo Han, Sunghyun Cho, Sunghyun Kwon, Soyoung Lee, Jun Ki Lee, Seung-Joon Yi, Byoung-Tak Zhang, Theo Taeyeong Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a new robot assistant to work in a busy warehouse. You don't just want a robot that can pick up a box once if you set it up perfectly. You want a robot that can work for eight hours straight, moving fast when it's safe, slowing down when it's tricky, and never needing you to step in and fix it every five minutes.

Most current robot brains (called Vision-Language-Action models) are like overly cautious students. They move very slowly, double-check everything, and if they make a tiny mistake, they freeze and wait for a human teacher to reset the scene. They are great at taking a test once, but terrible at working a full shift.

The paper introduces Habilis-β, a new robot brain designed to be a hardworking, fast-paced factory veteran. Here is how it works, explained through simple analogies:

1. The Problem: The "Perfect Test" vs. The "Real Shift"

Current robots are evaluated like students taking a single exam. If they pass, they get an A. But in the real world, robots don't take one exam; they take thousands of tests back-to-back.

The Issue: If a robot moves too slowly, it gets nothing done (low productivity). If it moves too fast without thinking, it crashes and needs a human to reset it (low reliability).
The New Metric: The authors created a "Productivity-Reliability Plane." Think of it as a scorecard with two numbers:
- TPH (Tasks Per Hour): How many boxes did you move? (Speed)
- MTBI (Mean Time Between Intervention): How long did you work before you had to call a human for help? (Reliability)
- Goal: You want high speed AND a long time between calls for help.

2. The Solution: How Habilis-β Learns

Habilis-β learns in three specific stages, like a student going from kindergarten to a master craftsman.

Stage 1: The "Playground" Phase (Learning to Move)

Before learning specific jobs, the robot is given unstructured "play" data.

The Analogy: Imagine a child playing with blocks, dropping them, picking them up, and stacking them randomly without a teacher telling them what to do.
Why it helps: The robot learns the physics of moving things. It learns how to recover if it drops a block or how to grab something awkwardly. This gives it a "gut feeling" for how objects behave, so it doesn't panic when things go slightly wrong.

Stage 2: The "Repetition" Phase (Learning to Endure)

Most robots are trained on "Success then Stop" videos. Habilis-β is trained on cyclic data.

The Analogy: Instead of watching a video of someone making one sandwich and stopping, Habilis-β watches a video of someone making sandwiches for 10 hours straight. It sees the bread get stale, the table get messy, and the person getting tired.
Why it helps: It learns to handle drift. In the real world, things aren't perfectly placed every time. This training teaches the robot to keep going even when the starting position is slightly off, preventing it from needing a human reset.

Stage 3: The "Speed Shaping" Phase (ESPADA)

This is the secret sauce for speed. The robot uses a technique called ESPADA (Spatially Aware Downsampling).

The Analogy: Imagine a video of a person walking across a room.
- Walking across the empty room: The robot learns to fast-forward this part. It doesn't need to see every step; it just needs to know "I'm going there."
- Picking up a fragile egg: The robot slows down to slow-motion. It needs to see every tiny movement to be precise.
Why it helps: It stops the robot from being slow and boring when it's safe to be fast. It compresses the "boring" parts of the movement so the robot can zip across the room and only focus on the tricky parts.

3. The Hardware: The "On-Device" Brain

Many robots rely on the cloud (the internet) to think. If the internet lags, the robot lags.

The Analogy: Habilis-β is like a smartphone with a powerful processor that does all the math right in your pocket. It doesn't need to ask Siri or Google for help.
Why it helps: Because it thinks locally, it can react instantly. It can make decisions in milliseconds, allowing it to correct mistakes before they become disasters.

4. The "Volume Knob" (CFG)

Finally, the system has a deployment-time knob called Classifier-Free Guidance (CFG).

The Analogy: Think of this as a volume knob for instructions.
- Low Volume: The robot relies more on its own "gut feeling" (the play data) to keep moving smoothly.
- High Volume: The robot follows the human's specific instructions very strictly, even if it means moving more aggressively.
Why it helps: You can tune the robot depending on the job. If you need it to be super precise, you turn the knob up. If you need it to be fast and flexible, you turn it down.

The Results: The "Factory Veteran"

When tested against other top robots (like $\pi_{0.5}$ and GR00T):

In Simulation: Habilis-β did 572 tasks per hour, while the others did about 120. It also went much longer without needing a human to fix it.
In the Real World: On a real humanoid robot doing a logistics job, Habilis-β completed 124 tasks per hour (compared to 19 for the others) and ran for 137 seconds between needing help (compared to 46 seconds for the others).

Summary

Habilis-β is a robot brain that stops acting like a nervous student taking a single test and starts acting like a seasoned worker. It learns by playing, it learns by working long shifts, it speeds up when it's safe, and it thinks fast enough to catch its own mistakes. It's designed not just to be "smart," but to be productive and reliable in the real world.

1. Problem Statement

Current Vision-Language-Action (VLA) models face significant hurdles in real-world industrial deployment, primarily due to three limitations:

Slow Motion: Many VLAs produce conservative, temporally dense actions that fail to utilize fast free-space motion, reducing throughput.
Lack of Long-Lasting Robustness: Standard evaluation relies on single-trial success rates with curated resets. This fails to capture performance degradation over time caused by state drift, error accumulation, and the need for frequent human intervention in continuous operation.
Latency Bottlenecks: Large VLA stacks often require heavy computing or networked components, making high-frequency, closed-loop control on edge devices difficult.

The authors argue that existing metrics (single-trial success rate) collapse the speed-accuracy trade-off and ignore long-run reliability. They propose a shift toward evaluating Fast-Motion (high speed when precision isn't critical), Long-Lasting (sustained operation without intervention), and On-Device (local inference for predictable latency) capabilities.

2. Methodology

The paper introduces Habilis-β, a VLA model designed to meet these deployment requirements through a three-stage training pipeline and specific architectural innovations.

A. Evaluation Framework: Productivity-Reliability Plane (PRP)

To address the evaluation gap, the authors propose the Productivity-Reliability Plane (PRP), using two metrics under a continuous-run protocol (1-hour autonomous cycles):

Tasks Per Hour (TPH): Measures productivity (throughput).
Mean Time Between Intervention (MTBI): Measures reliability (average time before human reset/recovery is needed).
Interventions include timeouts (failure to complete within a limit) and aborts (operator-initiated stops).

B. Training Pipeline (Three Stages)

Stage 1: Play Data Pre-training (Robust Priors):
- The model is pre-trained on large-scale, language-unconditioned "play" data.
- This data consists of unstructured teleoperated interactions without explicit task labels.
- Goal: Learn a task-agnostic, robust interaction prior that covers diverse physical behaviors (re-grasping, recovery, repositioning) often missing in success-only datasets.
Stage 2: Cyclic Data Post-training (Continuity):
- The model is fine-tuned on cyclic task demonstrations where the robot repeatedly executes the same task without resetting after success.
- Goal: Teach the model to handle state drift between cycles and learn recovery behaviors, aligning the training distribution with continuous real-world operation.
Stage 3: Distillation (Efficiency):
- A multi-step Flow Matching "teacher" model is distilled into a Rectified Flow "student" model.
- Goal: Compress the generation process from >10 steps to 2 steps, drastically reducing inference latency to enable high-frequency control on edge devices.

C. Key Technical Components

Architecture: A prefix-suffix design. A pre-trained VLM (Vision-Language Model) processes multimodal inputs (images + instruction) to condition a suffix Action Expert that generates continuous action chunks.
ESPADA (Spatially Aware Downsampling):
- A method to accelerate execution by semantically segmenting trajectories into "casual" (free-space transit) and "precision" (contact-rich) phases.
- It aggressively downsamples casual segments while preserving high-frequency control in precision phases, increasing throughput without sacrificing task success.
High-Frequency Control:
- The latency savings from distillation are reinvested into shorter action chunks and more frequent re-inference, improving closed-loop reactivity to unexpected events.
Classifier-Free Guidance (CFG):
- Used at inference time as a "knob" to balance instruction adherence vs. learned priors.
- By mixing instruction-conditioned and instruction-free predictions, operators can tune the policy to be more proactive (faster) or more conservative (safer) depending on the task.

D. Data Collection

The authors utilized a unified pipeline including:

Universal Data Device (UDD): A modular device using the actual deployment gripper to minimize embodiment mismatch.
Meta Quest Hand Tracking: For upper-body teleoperation.
Leader-Arm Teleoperation: For robot-native demonstrations.

3. Key Contributions

New Evaluation Paradigm: Introduction of the PRP (TPH and MTBI) metrics and continuous-run protocols that better reflect industrial deployment needs than single-trial success rates.
Habilis-β Model: A VLA system achieving state-of-the-art performance in both simulation and real-world humanoid logistics tasks.
Data Strategy: Demonstrating the efficacy of combining Play Data (for robust priors) and Cyclic Data (for continuity) to solve the distribution shift problem in long-horizon tasks.
Efficiency Innovations: Integration of Rectified Flow Distillation and ESPADA to enable high-frequency, on-device control with reduced latency.

4. Experimental Results

The authors evaluated Habilis-β against strong baselines ( $\pi0.5$ and GR00T N1.5) in both simulation (RoboTwin 2.0) and a real-world humanoid logistics workflow.

Simulation Results (1-hour continuous run):

Habilis-β: 572.6 TPH, 39.2s MTBI, 78.5% Success Rate.
$\pi0.5$ (Baseline): 120.5 TPH, 30.5s MTBI, 47.8% Success Rate.
Improvement: ~4.7x increase in throughput and significantly higher reliability.

Real-World Results (Humanoid RB-Y1, Dual-Bin Conveyor Packing):

Habilis-β: 124 TPH, 137.4s MTBI, 82.7% Success Rate.
$\pi0.5$ (Baseline): 19 TPH, 46.1s MTBI, 19.6% Success Rate.
Improvement: 6.53x improvement in productivity and 2.98x improvement in reliability.

Standard Benchmarks:

Habilis-β achieved the highest reported success rates on the standard RoboTwin 2.0 leaderboard for single-trial tasks (Dump Bin, Place Shoes, Stack Bowls), validating that the continuous-run optimizations did not degrade single-shot performance.

5. Significance and Impact

Bridging the Gap: The paper successfully bridges the gap between academic VLA research (focused on single-shot success) and industrial reality (focused on sustained throughput and reliability).
Scalable Deployment: By proving that on-device, high-frequency control is feasible with distilled models, it paves the way for deploying complex VLA policies on resource-constrained edge hardware without relying on cloud latency.
Data-Centric Insights: It highlights that the structure of training data (cyclic vs. episodic, play vs. task) is as critical as the model architecture for achieving robust, long-lasting robot behavior.
Practical Utility: The results demonstrate that robots can operate autonomously for hours in dynamic, cluttered environments with minimal human intervention, a critical step toward viable robotic automation in logistics and manufacturing.