🤖 AI

Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think

This paper introduces a training-free structural compression pipeline that identifies and removes redundant layers in Vision-Language-Action models, achieving up to 50% depth reduction and significant acceleration in both training and inference while maintaining or improving performance across diverse robotic tasks.

Original authors: Gia-Binh Nguyen, Trong-Bao Ho, Thien-Loc Ha, Khoa Vo, Philip Lund Møller, Quang T. Nguyen, Long Dinh, Tuan Dam, Vu Duong, Tung M. Luu, Trung Le, Tran Nguyen Le, Minh Vu, An Thai Le, Ngan Le, Daniel So

Published 2026-06-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Gia-Binh Nguyen, Trong-Bao Ho, Thien-Loc Ha, Khoa Vo, Philip Lund Møller, Quang T. Nguyen, Long Dinh, Tuan Dam, Vu Duong, Tung M. Luu, Trung Le, Tran Nguyen Le, Minh Vu, An Thai Le, Ngan Le, Daniel Sonntag, James Zou, Jan Peters, Duy M. H. Nguyen, Ngo Anh Vien

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, super-intelligent robot brain. This brain is a "Vision-Language-Action" (VLA) model. It can see the world, understand your spoken instructions, and figure out exactly how to move its arms to do things like fold a shirt or pick up a cup.

The problem is, these brains are gigantic. They are so big that training them takes days of supercomputer time, and running them in real-time is like trying to drive a Ferrari through a crowded market—it's slow, expensive, and requires a lot of fuel (computing power).

The authors of this paper asked a simple question: "Do we actually need all of that brain?"

The Big Discovery: The "Echo Chamber" Effect

The researchers looked inside these giant robot brains and found something surprising. They discovered that these brains are full of redundancy.

Think of the brain as a long assembly line with 50 workers (layers).

Worker 1 looks at the object.
Worker 2 looks at it again and says, "Yep, it's a cup."
Worker 3 looks at it and says, "Still a cup."
Worker 4 says, "Definitely a cup."

The researchers found that for long stretches of the assembly line, the workers are just repeating what the previous worker said. They aren't adding anything new; they are just echoing the same information. In technical terms, they found that consecutive layers produce nearly identical "thoughts" (representations).

The Solution: The "CKA" Scissors

Instead of keeping all 50 workers, the team invented a method called CLP (CKA-guided Layer Pruning).

Imagine you have a pair of smart scissors that can listen to the assembly line. The scissors use a tool called Centered Kernel Alignment (CKA) to measure how much two workers are saying the same thing.

If Worker 5 and Worker 6 are saying the exact same thing, the scissors snip them out.
The line is reconnected so Worker 4 passes the message directly to Worker 7.

Crucially, this doesn't require retraining the whole brain from scratch. It's a one-time "trim" done before the robot learns a new specific task.

The Results: Smaller, Faster, and Smarter

After trimming the fat, the results were impressive:

The Brain Got Smaller: They removed up to 50% of the layers. It's like turning a 50-story skyscraper into a 25-story one, but the building still stands just as strong.
Training Got Faster: Because the model is smaller, teaching it a new task took 40–50% less time. If a task used to take 20 hours to learn, it now takes about 10.
Real-Time Speed: The robot can now think and move about 30% faster in real life.
Better Performance in Some Cases: Surprisingly, in situations where the robot didn't have much data to learn from (like only seeing a task 100 times), the smaller brain actually performed better than the giant one.
- The Analogy: Think of it like studying for a test. If you have a massive, confusing textbook (the big model), you might get overwhelmed and memorize the wrong things (overfitting). If you have a concise, clear summary (the pruned model), you focus on the most important facts and do better on the test.

Where They Tested It

They didn't just test this on a computer simulation. They tried it on:

Virtual Robots: In video game worlds like LIBERO and RoboCasa.
Real Robots: Actual physical arms (like the UR10 and ALOHA robots) in a real lab, doing tasks like folding shorts, serving napkins, and stacking blocks.

The Bottom Line

The paper proves that these advanced robot brains are "over-engineered." They have way more layers than they actually need to do the job. By using a simple, training-free method to cut out the repetitive layers, we can make robots that are cheaper to train, faster to run, and just as good (or even better) at their jobs.

In short: You don't need a 50-layer brain to fold a shirt; a streamlined 25-layer brain does the job just fine, and it does it faster.

Technical Summary: Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think

Problem Statement

Vision-Language-Action (VLA) models, such as $\pi_0$ and GR00T-N1.5, have revolutionized robotic manipulation by leveraging massive pre-training on video-robot datasets to generate continuous control policies. However, these state-of-the-art (SOTA) architectures typically span billions of parameters, creating severe computational bottlenecks. These bottlenecks manifest as prohibitive hardware costs during downstream fine-tuning, high operational latency, and heavy memory-bandwidth overhead that constrain real-time edge inference.

Existing optimization strategies face three critical limitations:

Narrow Scope: Many methods target older, text-token autoregressive baselines (e.g., OpenVLA) rather than modern continuous-control foundations.
Training Inefficiency: Training-free methods (e.g., token pruning, caching) often accelerate inference but fail to reduce the computational cost of the expensive downstream fine-tuning phase.
Architectural Complexity: Training-adaptive methods (e.g., dynamic routing, early exiting) introduce auxiliary modules and complex training objectives, altering the core model structure and creating friction with downstream learning algorithms.

The central question addressed is whether redundant layers can be removed before fine-tuning without sacrificing policy performance.

Methodology: CKA-Guided Layer Pruning (CLP)

The authors propose CLP, a structural compression pipeline that is entirely training-free and requires no auxiliary routing modules. The core insight is that modern VLA backbones exhibit severe layer-wise representational redundancy, where consecutive transformer blocks produce highly correlated token representations.

The CLP framework operates as follows:

Redundancy Diagnosis: Using a compact calibration set sampled from training episodes, the method performs a single forward pass to extract hidden state representations.
Similarity Measurement: It employs Centered Kernel Alignment (CKA) to quantify the representational similarity between adjacent layers. CKA is chosen for its invariance to orthogonal transformations and isotropic scaling, making it robust for identifying structural redundancies.
Layer Identification: High CKA scores between consecutive layers indicate minimal feature transformation. The method aggregates adjacent redundant layers into contiguous blocks.
Static Pruning: Within each redundant block, all layers except the initial "functional anchor" layer are permanently removed. This process is applied independently to both the Vision-Language Model (VLM) backbone and the continuous control policy head.
Downstream Adaptation: The resulting compressed model is fine-tuned on the target task using the native training objective. No distillation losses or architectural modifications are required.

Key Contributions

Accessible Adaptation of SOTA VLAs: The paper demonstrates that SOTA continuous-control foundations ( $\pi_0$ , GR00T-N1.5) can be fine-tuned with substantially reduced depth, lowering memory, training, and inference costs without introducing auxiliary routing or early-exit modules.
Pre-Finetuning Backbone Compression: The introduction of CLP, a simple calibration-based framework that identifies and removes representationally redundant transformer blocks using CKA prior to downstream adaptation.
Diversified Multi-Embodiment Validation: Extensive evaluations across three simulation benchmarks (LIBERO, RoboCasa, SimplerEnv) and 10 real-world manipulation tasks spanning four robotic embodiments (UR10, UR5, single-arm ALOHA, bimanual ALOHA).

Experimental Results

The authors validate CLP across simulation and real-world settings, reporting the following key findings:

Compression Efficiency: CLP reduces model depth by up to 50% and trainable parameters by 25–37% across different backbones ( $\pi_0$ $π_{0}$ , GR00T-N1.5, SmolVLA).
- Training Speed: Achieves a 40–50% reduction in training time (e.g., 1.94 $\times$ faster on real-world tasks).
- Inference Speed: Delivers up to 30% faster real-time inference.
- FLOPs: Reduces computational cost by up to 50% (e.g., 49.3% reduction for GR00T-N1.5).
Performance Retention: Despite significant pruning, the streamlined architecture matches or exceeds the performance of full-scale base models.
- In simulation (LIBERO), CLP maintains competitive success rates while reducing trainable parameters by ~30%.
- In real-world deployments, CLP improves average success rates (e.g., from 73.5% to 75.9% on GR00T-N1.5) and outperforms full models on specific tasks like napkin serving (+20%) and screwdriver placement (+15%).
Regularization Effect: In data-scarce regimes, structural compression acts as an effective regularizer. For instance, restricting training to 10% of LIBERO data lifted success rates from 77.7% to 84.6% with CLP, outperforming both the full baseline and dynamic routing methods like MoLe-VLA.
Comparison with Baselines: CLP outperforms training-free token pruning methods (which do not reduce fine-tuning costs) and training-adaptive methods (which add complexity) in both efficiency and sample efficiency.

Significance and Claims

The paper claims that advanced VLAs require significantly fewer layers than previously assumed. By revealing a "highly non-trivial architectural characteristic" of severe layer-wise redundancy, the work offers a highly compute-efficient paradigm for scalable robot learning.

The authors emphasize that their approach is "structurally clean," avoiding the friction of dynamic routing or distillation pipelines. They argue that removing redundant layers not only accelerates training and inference but also implicitly regularizes the model, preventing overfitting in low-data regimes. The findings suggest a shift away from the assumption that ever-growing VLA architectures are necessary, pointing instead toward more efficient, sustainable foundation policies that can be adapted with minimal computational overhead.