🤖 AI

TempoVLA: Learning Speed-Controllable Vision-Language-Action Policies

TempoVLA introduces a speed-controllable Vision-Language-Action policy that leverages Variable-Speed Trajectory Augmentation (VSTA) and explicit speed conditioning to enable robots to dynamically adjust their execution speed for both faster transit and slower, precise contact phases, thereby overcoming the fixed-speed limitations of existing models.

Original authors: Dong Jing, Jingchen Nie, Tianqi Zhang, Jiaqi Liu, Huaxiu Yao, Zhiwu Lu, Mingyu Ding

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Dong Jing, Jingchen Nie, Tianqi Zhang, Jiaqi Liu, Huaxiu Yao, Zhiwu Lu, Mingyu Ding

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a robot that learned how to do a task, like stacking cups or folding a towel, by watching a human do it once. The problem is, that robot is stuck in a "one-speed" world. If the human demo was slow, the robot moves slowly forever. If the human demo was fast, the robot zooms around forever. It can't decide to slow down for a delicate part of the job or speed up when it's just moving through empty space.

TempoVLA is a new system that gives robots a "volume knob" for their speed. It allows a single robot brain to perform the exact same task at 0.5x speed (super slow and careful), 1x speed (normal), or 2x speed (super fast), all on command.

Here is how they did it, broken down into two simple parts:

1. The "Video Editor" (Data Side)

Before teaching the robot, the researchers had to fix the training videos. They created a clever tool called VSTA (Variable-Speed Trajectory Augmentation).

Think of a robot demonstration like a movie.

To make it faster: The editor takes a bunch of consecutive frames and merges them into one big jump. It's like skipping the boring parts of a movie to get to the action.
To make it slower: The editor takes one big movement and splits it into many tiny, slow steps. It's like adding "slow-motion" effects to a specific scene.

Crucially, this editing doesn't change what the robot is doing (the semantics), only how fast it does it. They took one original demonstration and turned it into a library of the same task done at six different speeds.

2. The "Speed Dial" (Model Side)

Once the robot has this library of fast and slow examples, they teach it a new trick. They give the robot a simple "speed dial" (a number like 0.5, 1.0, or 1.5) that it can see while it's working.

If you tell the robot, "Do this at 1.5x speed," the robot looks at its training data, sees the fast examples, and predicts bigger, bolder moves.
If you say, "Do this at 0.5x speed," it looks at the slow examples and predicts tiny, cautious moves.

The robot doesn't need to be retrained from scratch for every new speed. It just learns to listen to the "speed dial" and adjust its muscle movements accordingly.

The "Smart Pilot" Bonus

The paper also shows that you can pair this robot with a "Smart Pilot" (a large AI language model). Instead of you manually typing "slow down," the Smart Pilot watches the robot's camera feed.

Scenario: The robot is reaching across an empty table.
- Smart Pilot: "Clear path! Go fast!" (Robot speeds up).
Scenario: The robot is about to grab a fragile cup.
- Smart Pilot: "Danger zone! Slow down!" (Robot slows to a crawl).

This creates a robot that naturally accelerates when it's safe and decelerates when it needs precision, all without human intervention.

What They Found

Flexibility: The robot can successfully switch between speeds on the fly.
Better Performance: Surprisingly, training the robot on these mixed speeds actually made it better at the normal speed (1x) than robots trained only on normal speed. It seems learning to move at different speeds helps the robot understand the task better.
Limits: There is a physical limit. If you ask the robot to go too fast (like 4x speed), the robot's physical motors and controllers can't keep up with the brain's instructions, and it starts to miss its target. But within a reasonable range (0.5x to 1.5x), it works perfectly.

In short, TempoVLA turns a rigid, single-speed robot into a flexible worker that can choose its own pace, making it safer for delicate tasks and faster for routine ones.

Technical Summary: TempoVLA

Problem Statement

Existing Vision-Language-Action (VLA) models for robot manipulation inherit a single, fixed execution speed from their training demonstrations. This rigidity is a significant limitation because real-world manipulation tasks alternate between low-risk transit phases (requiring fast execution) and high-risk contact stages (demanding slow, precise motion). Current efforts to alter execution speed—such as model compression, KV-cache reuse, or reinforcement learning—typically shift the policy to a different fixed speed rather than enabling explicit, on-demand control. Furthermore, these methods focus almost exclusively on acceleration, leaving deceleration (essential for precision insertion and fragile handovers) largely unexplored. The core challenge is to equip a single VLA with explicit, bidirectional speed control without retraining its base architecture from scratch or collecting new data.

Methodology

The authors propose TempoVLA, a framework that enables a single VLA policy to execute tasks at variable speeds controlled by an explicit scalar condition $s$ . The approach consists of two coupled components: a data-side augmentation strategy and a model-side conditioning mechanism.

1. Variable-Speed Trajectory Augmentation (VSTA)

VSTA is an online data augmentation strategy that re-times any existing demonstration to a target speed $s$ while preserving motion semantics. It operates in three steps:

Motion-Consistent Segmentation: Demonstrations are segmented based on motion modes (still, translate, rotate, or translate-and-rotate) and gripper events. Boundaries are placed at mode changes or direction reversals to ensure semantic consistency.
Chunk-Level Speed Transform: Within each segment, the target speed $s$ is realized by mapping $q$ source frames to $p$ output frames ( $s = q/p$ ). The total motion $\Delta$ of a chunk is accumulated and then re-split into $p$ equal-magnitude steps via linear interpolation. This preserves the integrated motion of the chunk while altering the within-chunk trajectory shape. This operation is valid for action spaces closed under addition (e.g., Cartesian translations, joint velocities, and axis-angle vectors).
Online Chunk-Start Sampling: To avoid discarding observations during the re-timing process, the starting offset of chunks is randomized for every training sample. This ensures that over the course of training, every source frame eventually serves as a valid chunk start, contributing to the training signal.

2. Speed Conditioning Mechanism

The policy $\pi_\theta(o_t, s)$ is trained on the augmented dataset with the speed scalar $s$ fed as an explicit conditioning input. The paper evaluates three lightweight injection schemes:

Textual Prefix: Prepending a phrase (e.g., "Perform the task at 1.5x speed") to the language instruction.
Speed-Modulated RMSNorm: Using a small MLP to embed $s$ and adding it to the timestep embedding, which then modulates the RMSNorm of the action expert layers.
Soft Prompt with Speed Anchors: Inserting learnable soft-prompt tokens corresponding to discrete speed anchors into the encoder input.

Experiments indicate that speed control can be injected with minimal engineering effort and is largely independent of the specific conditioning mechanism, with the textual prefix chosen as the default for its simplicity and flexibility.

3. Dynamic Speed Control with VLM Scheduler

Beyond fixed commands, TempoVLA supports automated dynamic speed scheduling when paired with a high-level Vision-Language Model (VLM). The VLM observes the scene and dispatches per-chunk speed commands, allowing the system to accelerate during low-risk transit and decelerate for high-risk contact phases without human intervention.

Key Results

Simulation (LIBERO Benchmark)

Feasibility: VSTA successfully generates executable demonstrations across a range of speeds (0.5x to 2x) with negligible motion error ( $< 5 \times 10^{-8}$ ).
Performance Gains: Training with VSTA improves the default 1x success rate compared to a single-speed baseline. The peak performance often occurs at moderate speeds (1.25x or 1.5x), suggesting that VSTA's compression of "rhythm padding" in teleoperation data leads to more decisive execution.
Granularity: Finer speed granularity in training (e.g., a 7-speed range) yields better performance across all speeds compared to coarser ranges.
Limits: At extreme speeds (e.g., 4x), performance degrades due to the low-level controller's inability to track large action magnitudes within a single control interval, not due to policy failure.

Real-World Experiments

Setup: Deployed on a 7-DoF Franka arm with a parallel gripper across five tasks (pick-and-place and deformable object manipulation).
Success Rate: VSTA boosted the default 1x success rate from 80.0% (baseline) to 88.0%.
Dynamic Scheduling: When paired with a GPT-4o scheduler, the system achieved a 96% average success rate, outperforming the best fixed-speed configuration. The scheduler correctly identified phases (free-space vs. contact) and adjusted speeds accordingly, though it remained conservative, rarely dispatching the maximum 1.5x speed.
Realized Speed: The realized execution speed closely tracked commanded speeds (e.g., 1.29x realized for 1.25x command), confirming faithful translation of policy predictions to hardware execution.

Significance and Claims

The paper claims that TempoVLA provides a lightweight, plug-and-play solution for bidirectional speed control in existing VLAs. Its primary contributions are:

Bidirectional Control: It enables both acceleration and deceleration from a single policy, addressing a gap in prior work that focused mainly on acceleration.
Data Efficiency: It achieves this without new data collection, using VSTA to re-time existing demonstrations.
Implicit Augmentation: The process of re-timing data acts as an effective augmentation that improves the default 1x performance, forcing the policy to learn finer object- and goal-aware features rather than memorizing fixed observation-to-magnitude mappings.
Dynamic Scheduling: It demonstrates that execution speed can serve as a new control channel for high-level reasoners (VLMs), enabling phase-aware execution where the robot autonomously balances speed and precision.

The authors emphasize that while the policy itself supports a wide range of speeds, the upper limit of realized speed is governed by the low-level controller's tracking bandwidth, suggesting that future work could involve co-tuning the controller to fully exploit the policy's speed-control envelope.