💻 computer science

How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf

This paper introduces VLA-Perf, an analytical performance model that systematically evaluates the inference performance of Vision-Language-Action models across various architectures and deployment scenarios to provide practical guidance for designing future real-time embodied AI systems.

Original authors: Wenqi Jiang, Jason Clemons, Karu Sankaralingam, Christos Kozyrakis

Published 2026-02-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Wenqi Jiang, Jason Clemons, Karu Sankaralingam, Christos Kozyrakis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have built a brilliant, super-smart robot brain. This brain can see the world through cameras, understand human language, and decide how to move its arms and legs. This is called a VLA (Vision-Language-Action) model.

But here's the problem: A robot brain is useless if it's too slow. If a robot sees a ball rolling toward it, it needs to move instantly. If the brain takes too long to think, the robot gets hit.

The paper you shared, "How Fast Can I Run My VLA?", is like a mechanic's manual for these robot brains. The authors from NVIDIA Research built a tool called VLA-Perf to answer one big question: "How do we make these robot brains fast enough to run in the real world?"

Here is the breakdown using simple analogies:

1. The Problem: The "Combinatorial Chaos"

Imagine you are trying to build the perfect delivery truck. You have to choose:

The Engine: (The AI Model) Should it be a tiny, fuel-efficient 4-cylinder or a massive V12?
The Driver: (The Inference System) Is the driver sitting in the truck (on-device), in a control tower nearby (edge server), or in a skyscraper in another city (cloud)?
The Road: (The Network) Is the driver on a super-highway (fiber optic), a country road (WiFi), or a bumpy dirt path (4G/5G)?

There are millions of ways to mix and match these parts. The authors realized nobody had tested them all. So, they built VLA-Perf, a "simulator" that predicts how fast any combination will go without needing to build the actual truck first.

2. The Key Findings (The 15 Takeaways Simplified)

The paper tested thousands of scenarios and found some surprising rules of the road. Here are the most important ones:

🚗 The Engine Size Matters (Model Scaling)

The Analogy: Think of the AI model as a chef. A small chef (small model) can chop vegetables quickly. A giant chef (huge model) makes better soup but takes forever to chop.
The Finding: If you want the robot to move at "real-time" speeds (like 10 to 100 times a second), you can't just keep making the chef bigger.
- Big Datacenter Servers (like a super-kitchen) can handle huge chefs and still be fast.
- Small Robot Chips (like a tiny kitchen in the robot's head) get overwhelmed if the chef is too big. They can only handle small, efficient chefs.

🎬 The "Long Memory" Trap (Long Context)

The Analogy: Imagine a robot trying to remember a movie it watched 10 minutes ago to decide what to do now.
The Finding: If the robot tries to remember too much (thousands of past frames), it gets bogged down.
- Powerful servers can remember a lot (up to 1,000 frames) and still be fast.
- Small robot chips can only remember about 100 frames before they start stuttering.

🎨 The Painting Style (Diffusion vs. Autoregressive)

The Analogy:
- Autoregressive: Like painting a picture one single brushstroke at a time, waiting for the paint to dry before the next stroke. Very slow.
- Diffusion: Like starting with a blurry image and sharpening it all at once in a few steps. Much faster.
The Finding: The "Diffusion" style (sharpening the image) is currently 100 times faster than the "Autoregressive" style (one stroke at a time) for robots. If you want speed, use Diffusion.

🏠 Where Should the Brain Live? (On-Device vs. Cloud)

The Analogy:
- On-Device: The brain is inside the robot's head. No travel time for thoughts, but the head is small and weak.
- Cloud: The brain is in a massive supercomputer far away. The brain is super smart, but the thoughts have to travel over the internet.
The Finding:
- Cloud is usually faster than the robot's own brain, unless the internet connection is terrible (like a slow 4G signal).
- If the internet is slow, it's better to have a weaker brain right in the robot's head than to wait for a message from a distant cloud.

🤝 The "Split Brain" Strategy (Collaboration)

The Analogy: What if the robot's head does the easy thinking, and the cloud does the hard thinking?
The Finding: This usually fails. It's like having a secretary send a memo to the CEO, wait for a reply, and then act. The time spent sending the memo back and forth is usually slower than just letting the robot's own brain do the whole job.

⚡ The "Async" Superpower (Doing Two Things at Once)

The Analogy: Imagine a waiter taking an order.
- Synchronous: The waiter waits for the kitchen to finish cooking before taking the next order.
- Asynchronous: The waiter takes the next order while the kitchen is still cooking the first one.
The Finding: If the robot is connected to a slow network, "Asynchronous" thinking is a game-changer. The robot can start moving based on "old" information while the new calculation is still traveling over the internet. This keeps the robot moving smoothly even on bad connections.

3. The Bottom Line: How to Build a Fast Robot

The authors give us a recipe for the future:

If you have a powerful server nearby: Put the big brain there. Use a fast internet connection (WiFi 7 or Ethernet). Use "Asynchronous" thinking to hide the internet lag.
If the robot must work alone (no internet): You need a small, efficient brain. You can't use the massive models. You might need to simplify the model or reduce the number of "steps" it takes to think.
Don't split the brain: It's usually better to keep the whole brain in one place (either all in the robot or all in the cloud) rather than splitting it up.

Summary

This paper is a guide for engineers. It says: "Stop guessing. Use our tool (VLA-Perf) to figure out exactly how big your robot's brain should be and where it should live, so your robot doesn't trip over its own feet because it was thinking too slowly."

It turns the complex math of AI into a simple rule: Match the brain size to the hardware, and match the location to the internet speed.

1. Problem Statement

Vision-Language-Action (VLA) models are central to Embodied AI, enabling robots to perceive, reason, and act. However, deploying these models on real-world robots faces a critical bottleneck: real-time inference constraints.

The Gap: While various techniques exist to optimize VLA efficiency (e.g., quantization, smaller models, asynchronous execution), there is no comprehensive understanding of the VLA inference performance landscape.
The Complexity: The performance is determined by a vast combinatorial space involving:
1. Model Architectures: Scaling laws, long-context handling, autoregressive vs. diffusion-based action experts, denoising steps, and action chunking.
2. Inference Systems: Hardware capabilities (Edge vs. Datacenter GPUs), deployment location (On-device, Edge-server, Cloud), and network conditions (Wired, WiFi, 4G/5G).
The Challenge: Exhaustively testing every combination of models and systems is cost-prohibitive and time-consuming. The paper asks: How should future VLA models and systems be designed to support real-time inference (10–100 Hz)?

2. Methodology: VLA-Perf

To address the evaluation challenge, the authors introduce VLA-Perf, an analytical, roofline-based performance model.

Core Concept: VLA-Perf predicts optimal inference latency and throughput for arbitrary model-system combinations without requiring physical deployment.
Modeling Approach:
- Components: It abstracts VLA inference into model components (Vision Encoder, VLM Backbone, Action Expert) interleaved with data transfers (local or networked).
- Latency Calculation: Total latency ( $T_{total}$ $T_{t o t a l}$ ) is the sum of model inference latency and data movement latency.
  - Compute/Memory Bound: Operator latency is calculated using a roofline model: $T_o = \max(\frac{FLOPs}{PeakFLOP/s}, \frac{Bytes}{MemBW})$ .
  - Network Latency: Modeled as $T_{net} = NetLat + \frac{Bytes}{NetBW}$ .
- Validation: The model was validated against real-world Triton inference benchmarks on an RTX 4090 using the $\pi_0$ model. It achieved 73.3%–82.6% fidelity compared to empirical measurements, narrowing the gap as workload size increases.
Evaluation Scope: The study evaluates the $\pi_0$ $π_{0}$ architecture (and its scaled variants) across:
- Hardware: Jetson Thor (Edge), RTX 4090 (Consumer), A100/H100/B100 (Datacenter).
- Networks: Ethernet (1G/10G), WiFi 6/7, 4G/5G, and Cloud configurations.
- Configurations: Model sizes (up to 81B params), context lengths (up to 10k steps), denoising steps, action chunk sizes, and synchronous/asynchronous execution.

3. Key Contributions & Results (15 Takeaways)

The study distills 15 actionable insights, categorized by model design and system deployment.

A. Model Design Insights

Hardware Limits: Datacenter GPUs (A100/B100) can run small VLAs (like $\pi_0$ ) at camera frame rates (24–60 Hz). Edge GPUs (Jetson Thor) are significantly slower (~19 Hz).
Memory vs. Compute: Action prediction is memory-bound across all hardware, while Vision and VLM components are compute-bound (except on Jetson Thor, where memory bandwidth limits everything).
Scaling Laws: Inference latency scales linearly with model size. Datacenter GPUs can support models 30x larger than $\pi_0$ (81B params) while maintaining real-time speeds; edge GPUs cannot.
Long-Context Feasibility: Datacenter GPUs support real-time inference with up to 1,000 past timesteps. Edge/Consumer GPUs are limited to ~100 steps before latency becomes prohibitive.
Denoising Steps: Increasing diffusion steps has a linear, significant impact on latency. Increasing action chunk size has a negligible effect because action prediction is memory-bound.
Architecture Choice: Diffusion-based action experts are 10–100x faster than vanilla autoregressive models when using action chunking.
Autoregressive Viability: Autoregressive models only become competitive with small action token counts or when parallel decoding is used (for small chunk sizes). For large chunks, they become compute-bound and slower.

B. System Deployment Insights

Server vs. Device: Server-side inference (even with consumer GPUs) significantly outperforms on-device inference in most scenarios, unless network conditions are extremely poor (e.g., slow 4G).
Collaborative Inference: Splitting work between device and server (running VLM on server, Action Expert on device) is generally slower than running entirely on the server or entirely on the device due to the overhead of transferring KV caches.
Asynchronous Inference: Asynchrony (overlapping inference with robot execution) drastically improves throughput for server-side systems, especially over slow wireless networks (2.6x–6x speedup), though it does not reduce end-to-end latency.
Dual-System Pipelines: A "System 1 (fast) + System 2 (slow)" architecture improves performance, but gains are highly dependent on network latency. On fast networks, it can double throughput; on slow networks, benefits vanish.
10 Hz Target: Achievable on Jetson Thor for $\pi_0$ and on Edge Servers with consumer GPUs + wireless networks.
100 Hz Target:
- On-Device: Requires model-level optimizations (smaller models, fewer steps).
- Edge/Cloud: Requires Datacenter GPUs (B100) and high-speed networks (Wired/WiFi 7).
Cloud Inference: Achieving 100 Hz in the cloud is impossible with synchronous inference due to network latency. Asynchronous inference is mandatory for high-frequency cloud deployment.
Network Criticality: Network performance is the primary bottleneck for server-side inference. Poor networks (4G/Slow Cloud) degrade synchronous performance to unusable levels (<10 Hz), while asynchronous execution can recover acceptable performance.

4. Significance

First Systematic Study: This is the first work to comprehensively map the VLA inference performance landscape, bridging the gap between model design and system deployment.
Practical Guidance: The 15 takeaways provide concrete rules for engineers:
- Designers should prioritize diffusion-based action experts, manage denoising steps carefully, and avoid over-reliance on long contexts for edge devices.
- System Architects should prioritize server-side deployment with high-bandwidth networks and utilize asynchronous execution to overcome network bottlenecks.
Open Source Tool: The release of VLA-Perf allows the community to simulate and optimize VLA systems without expensive hardware prototyping, accelerating the development of real-time embodied AI.

Conclusion

The paper concludes that while VLA models are powerful, achieving real-time performance requires a co-design of model architecture and system infrastructure. Datacenter GPUs with asynchronous, server-side inference currently offer the best path to high-frequency (100 Hz) operation, while edge deployment requires aggressive model optimization and acceptance of lower frequencies (10–20 Hz). Future work must address diverse embodied domains (e.g., autonomous driving) and integrate sensing/actuation latencies into the performance model.