FurnitureVLA: Learning Long-Horizon Bimanual Furniture Assembly with Vision-Language-Action Model
This paper introduces FurnitureVLA, a Vision-Language-Action model that enables real-scale, long-horizon bimanual furniture assembly by leveraging a progress-enhanced architecture and high-quality teleoperated demonstrations to achieve significant success rate improvements over baselines in both simulation and real-world validation.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine trying to build a complex piece of furniture, like an IKEA chair, but instead of using your hands, you are controlling two giant robotic arms. Now, imagine doing this without a manual, just by watching a video and trying to guess the next move. That is the challenge this paper tackles.
The researchers created a new system called FurnitureVLA to teach robots how to assemble real-sized furniture using two arms at once. Here is how they did it, broken down into simple concepts:
1. The Problem: A Robot's "Long Memory" Issue
Most robots are good at short tasks, like picking up a cup and putting it down. But building furniture is a "long-horizon" task. It's like trying to write a whole novel in one breath. If the robot makes a tiny mistake in step 1 (like placing a leg slightly crooked), that error gets bigger and bigger by step 10, causing the whole thing to collapse.
Previous attempts only worked on tiny, toy-sized furniture or with just one robotic arm. This paper is the first to try real-sized furniture with two arms working together.
2. The Solution: Breaking the Movie into Scenes
To stop the robot from getting confused, the researchers didn't teach it to build the whole chair in one go. Instead, they broke the job down into small, manageable "scenes" or subtasks, like a movie script.
- Scene 1: Pick up the left leg.
- Scene 2: Insert the left leg.
- Scene 3: Step back.
They used a VR Teleoperation System to collect the data. Think of this as a human playing a video game where they wear a headset and control two robotic arms with their own hands. This allowed them to collect high-quality "demonstrations" of how a human expert would build the furniture.
3. The Secret Sauce: The "Progress Meter"
The biggest innovation is something they call a Progress-Enhanced VLA.
Imagine you are driving a car on a long trip. If you just tell the car "Drive to New York," it might get lost after 50 miles. But if you tell it, "Drive to the next gas station," then "Drive to the next town," it stays on track.
The researchers gave the robot a continuous progress meter (a signal that goes from 0% to 100% for each step).
- As the robot works, it constantly checks its progress.
- When the meter hits 100%, the robot automatically knows, "Okay, I'm done with this step. Now I switch to the next instruction."
- This prevents the robot from getting stuck or confused about which part of the assembly it is currently doing.
4. The Training Ground: A Digital Sandbox
Before testing on real metal arms, they built a massive simulation pipeline.
- They created a digital world where they could generate thousands of perfect "expert" builds using computer algorithms.
- They tested three different pieces of furniture: a small side table (easy), a shelf (medium), and a complex chair (hard).
- The chair task was the hardest, requiring 1,550 individual control steps (like 1,550 tiny movements) to finish.
5. The Results: From Failing to Succeeding
- Without their new method: The robot failed almost 100% of the time on the complex chair. It was like a student trying to solve a calculus problem without knowing basic math.
- With FurnitureVLA: They boosted the success rate from 48% to 80% in the simulation.
- Real World Test: They tested it on a real robot in a lab. Even with the messy real world (lighting, friction, slight errors), the robot only dropped about 16% in performance compared to the simulation. It successfully assembled the complex chair most of the time.
6. What Made It Work? (The Fine-Tuning)
The researchers also tested different "knobs" to see what made the robot more precise:
- More Cameras: Adding a camera at the back helped the robot see parts that were hidden from the front.
- Smoothing the Motion: They found that averaging the robot's planned moves over a few seconds (like smoothing out a shaky video) helped it handle heavy furniture better.
- Higher Resolution: Giving the robot sharper eyes (higher image quality) helped it align the tiny holes and magnets perfectly.
Summary
In short, the paper introduces a robot that can build real furniture by:
- Learning from human demonstrations done in VR.
- Breaking the huge job into tiny, manageable steps.
- Using a "progress meter" to know exactly when to switch to the next step.
- Fine-tuning its vision and movement to be precise enough to snap parts together without breaking them.
It's a major step forward in teaching robots to do complex, multi-step household chores that require two hands and a lot of patience.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.