💻 computer science

Data Scaling Laws in Imitation Learning for Robotic Manipulation

This paper demonstrates that in robotic manipulation, generalization performance follows power-law scaling with the diversity of training environments and objects rather than the sheer volume of demonstrations, enabling highly effective zero-shot policies through a focused data collection strategy.

Original authors: Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, Yang Gao

Published 2026-06-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, Yang Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to do a specific chore, like pouring water from a bottle into a cup. In the past, you might have thought the best way to teach it was to show the robot the exact same action thousands of times in the exact same kitchen. You'd think, "If I just give it enough practice, it will get perfect."

This paper says: No, that's not the most efficient way.

Instead of practicing the same move over and over, you should teach the robot to do that same move in many different kitchens with many different bottles.

Here is the breakdown of their findings using simple analogies:

1. The "Variety" vs. "Volume" Rule

The researchers discovered a surprising law about how robots learn. They found that variety is far more important than volume.

The Old Way (Volume): Showing a robot 1,000 videos of pouring water in one specific kitchen with one specific blue bottle.
The New Way (Variety): Showing a robot just 50 videos of pouring water, but in 50 different kitchens, using 50 different types of bottles (glass, plastic, tall, short, red, clear).

The Analogy: Think of it like learning to drive.

If you only practice driving in your driveway on a sunny day (high volume, low variety), you will crash the moment you hit a rainy street or a busy intersection.
If you practice driving in 32 different neighborhoods, in rain, snow, and traffic, but only drive for a short time in each (high variety, lower volume), you will become a much better driver much faster.

The paper found that once you have enough different scenarios (about 32 different environments), adding more practice videos for the same scenario barely helps at all.

2. The "Power Law" of Learning

The researchers found that the robot's ability to handle new, unseen situations follows a predictable mathematical curve called a Power Law.

The Analogy: Imagine the robot's "generalization skill" is a muscle.

If you lift a weight (add a new environment or object) once, your muscle grows a little.
If you lift weights in 2 different gyms, it grows more.
If you lift weights in 32 different gyms, your muscle becomes incredibly strong.
The paper shows that this growth isn't random; it follows a smooth, predictable line. The more different places and objects you train on, the better the robot gets at handling new places and objects it has never seen before.

3. The "One Afternoon" Breakthrough

The most exciting part of the paper is how efficient this method is.

The researchers tested this on four different tasks: pouring water, arranging a computer mouse, folding towels, and unplugging a charger.

They used four people (data collectors).
They worked for just one afternoon.
They collected data in 32 different environments with 50 demonstrations for each.

The Result: They trained robots that could perform these tasks with 90% success rates in completely new rooms with objects they had never seen before. They didn't need to fine-tune the robot or retrain it; it just worked immediately (zero-shot).

4. What About the Robot's "Brain"?

The paper also looked at the size of the robot's "brain" (the AI model).

The Visual Encoder (The Eyes): Making the robot's "eyes" bigger and smarter (using a larger model) definitely helped it see better and perform better.
The Action Model (The Hands): Surprisingly, making the "hand" part of the brain bigger didn't help. A smaller, simpler model was just as good as a giant one for these specific tasks. It seems the "hands" didn't need to be more complex; they just needed to have seen more variety.

The Bottom Line

The paper argues that to build a robot that can handle the real world, we shouldn't just dump massive amounts of repetitive data into it. Instead, we should focus on diversity.

If you want a robot that can pour water into any cup in any house, don't show it 10,000 videos of pouring in one kitchen. Show it 1,600 videos (32 locations × 50 tries) across 32 different kitchens with 32 different cups. That small, diverse dataset is the "secret sauce" to making robots truly adaptable.

Technical Summary: Data Scaling Laws in Imitation Learning for Robotic Manipulation

Problem Statement

While data scaling has revolutionized Natural Language Processing (NLP) and Computer Vision (CV), leading to models with remarkable generalization capabilities, robotics—specifically robotic manipulation—lacks established scaling laws. Current robotic policies often struggle with zero-shot generalization to unseen objects and environments, typically requiring fine-tuning or extensive task-specific data. This paper investigates whether similar data scaling laws exist in robotic imitation learning. The central question is: Can appropriate data scaling produce single-task robot policies capable of zero-shot deployment on nearly any object within the same category, in any environment?

Methodology

Experimental Setup

The authors conducted a comprehensive empirical study using Imitation Learning (IL) via Behavior Cloning (BC).

Tasks: The study focused on four manipulation tasks: Pour Water, Mouse Arrangement, Fold Towels, and Unplug Charger.
Data Collection: Using the Universal Manipulation Interface (UMI), a hand-held gripper, the team collected over 40,000 human demonstrations. Data was gathered across numerous "in-the-wild" environments and with diverse objects within the same category.
Policy Architecture: The authors employed Diffusion Policy with a CNN-based U-Net. Key architectural choices included:
- Visual Encoder: A DINOv2 ViT-Large/14, which was fully fine-tuned (outperforming ImageNet ResNet and CLIP ViT).
- Temporal Ensemble: An exponential weighting scheme to smooth action transitions and reduce motion discontinuity.
Evaluation Protocol: A rigorous evaluation was conducted on unseen environments and objects. Performance was measured using a normalized score (0 to 1) based on tester-assigned step-wise criteria, rather than sparse binary success rates. The study involved over 15,000 real-world robot rollouts.

Scaling Dimensions

The study systematically varied three dimensions to analyze their impact on generalization:

Number of Training Environments ( $M$ ): Varying the diversity of scenes (lighting, distractors, backgrounds).
Number of Training Objects ( $N$ ): Varying the diversity of objects (color, size, geometry) within the same category.
Number of Demonstrations ( $K$ ): Varying the quantity of data per environment-object pair.

Key Findings and Results

1. Power-Law Scaling for Diversity

The generalization performance of the policy follows a roughly power-law relationship with the number of training environments, objects, and environment-object pairs.

Object Generalization: Performance on unseen objects improves steeply with the number of training objects. With only 8 training objects, normalized scores exceeded 0.8; with 32 objects, scores surpassed 0.9.
Environment Generalization: Performance on unseen environments also follows a power law, though the initial slope is slightly less steep than for objects.
Combined Generalization: When varying both environments and objects simultaneously, the policy generalizes effectively to novel combinations.

2. Diversity Over Quantity

A critical finding is that diversity is far more important than the absolute number of demonstrations.

Diminishing Returns: Once the number of demonstrations per environment or object reaches a certain threshold, additional data yields minimal performance gains.
Saturation Points:
- For 8 environment-object pairs, performance saturates at ~400 total demonstrations.
- For 16 pairs, saturation occurs at ~800.
- For 32 pairs, saturation occurs at ~1,600.
Efficiency: Increasing the number of environments and objects is significantly more effective than increasing the number of demonstrations per specific environment-object pair.

3. Efficient Data Collection Strategy

Based on the scaling laws, the authors propose a specific data collection strategy:

Strategy: Collect data in as many diverse environments as possible, with one unique object per environment.
Threshold: A total of 32 environment-object pairs is generally sufficient to train a policy that generalizes well.
Demonstrations per Pair: Collecting 50 demonstrations per environment-object pair is recommended for tasks of similar difficulty.
Validation: Applying this strategy (32 pairs, 50 demos each) to two new tasks (Fold Towels and Unplug Charger) using only four data collectors for one afternoon resulted in policies achieving ~90% success rates in novel environments with unseen objects.

4. Model Scaling Insights

Beyond data, the paper explored model scaling:

Visual Encoder: Scaling the visual encoder (from ViT-Small to ViT-Large) and fully fine-tuning pre-trained DINOv2 features are essential. Learning from scratch or using frozen features resulted in near-zero performance.
Action Diffusion Model: Surprisingly, scaling the size of the U-Net action diffusion model (from small to large) did not improve performance and slightly degraded it in the largest configuration, suggesting the smaller model capacity was already sufficient for the action distribution.

Significance and Claims

The paper claims to establish the first comprehensive data scaling laws for robotic manipulation, shifting the focus from "more data" to "better data diversity."

Generalization is Achievable: The authors demonstrate that "in-the-wild" generalization is closer than previously thought. A single-task policy can be trained to operate on any object in any environment without fine-tuning, provided the training data covers sufficient environmental and object diversity.
Cost-Effectiveness: The proposed strategy significantly reduces the time and labor required to build robust robotic policies. Achieving ~90% success rates required only a single afternoon of data collection by four people, challenging the notion that robotic generalization requires massive, years-long data collection efforts.
Foundation for Future Work: By identifying that diversity drives generalization more than volume, the paper provides a blueprint for constructing large-scale robotic datasets that prioritize environmental and object variety over sheer trajectory count.

The authors acknowledge limitations, noting that their study is restricted to single-task policies (not task-level generalization), imitation learning (not reinforcement learning), and a specific set of four tasks. They call for future work to verify these laws across a broader range of tasks and algorithms.