From Vocal Instructions to Household Tasks: The Inria TIAGo++ in the euROBIN Service Robots Coopetition
This paper presents the Inria team's open-source robotics system, featuring a modified TIAGo++ platform with whole-body control and an LLM-based planning pipeline, which successfully demonstrated voice-activated household task execution and custom teleoperation during the 1st euROBIN coopetition.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a high-stakes cooking competition, but instead of human chefs, the contestants are robots. The challenge? They have to listen to a referee's voice commands, walk into a messy kitchen, find specific items, and hand them to a person—all without dropping anything or getting stuck.
This paper is the "victory lap" report from the Inria team (a group of French researchers) who built a robot called TIAGo++ to compete in this event, known as the euROBIN coopetition. "Coopetition" is a fun mix of "cooperation" and "competition"—teams compete to win, but they also share their best software tricks to help everyone improve.
Here is a breakdown of how their robot worked, using simple analogies:
1. The Brain: The "Smart Sous-Chef" (LLM)
The robot doesn't just follow a rigid script like a microwave. Instead, it has a Large Language Model (LLM) acting as its brain.
- The Analogy: Think of the LLM as a very smart, experienced sous-chef who speaks human language. When the referee says, "Please grab the red cup and give it to the tall man," the robot doesn't just hear words; it understands the intent.
- How it works: The robot listens to the voice, converts it to text, and the "sous-chef" brain turns that sentence into a step-by-step recipe (a JSON plan). It even explains its reasoning out loud ("I am going to the table first...") so the humans know what it's thinking. To make sure it doesn't get confused or make up fake steps (hallucinations), the researchers built a strict "grammar guard" that forces the robot to stick to the rules.
2. The Body: The "Acrobatic Dancer" (Whole-Body Control)
The robot has a mobile base (wheels), two arms, and a torso. Moving all these parts at once is like trying to juggle while riding a unicycle.
- The Analogy: The Whole-Body Control (WBC) system is like a highly trained dance instructor. It ensures that when the robot reaches for a cup with its left hand, its right arm doesn't bump into a cabinet, and its wheels don't spin out of control. It calculates the perfect movement for every single joint simultaneously to keep the robot balanced and safe.
3. The Eyes: The "Detective with a Flashlight" (Perception)
To find objects in a messy kitchen, the robot needs to see clearly.
- The Analogy: The robot uses special 3D cameras (RGB-D) like a detective with a flashlight that can see depth. Instead of just seeing a "cup," it sees a "cup located 2 meters away, tilted at a 15-degree angle."
- The Trick: To make things easier, they placed special QR-code-like stickers (called AprilTags) on objects and locations. It's like putting a GPS beacon on the cup so the robot never gets lost looking for it.
4. The Hands: The "Shadow Puppeteer" (Teleoperation)
Sometimes, the robot gets stuck or the kitchen is too messy for its pre-programmed skills. That's when the humans step in.
- The Analogy: The team built custom controllers that look like small, lightweight robot arms. When a human moves their controller, the big robot mimics the movement exactly, like a shadow puppeteer controlling a shadow.
- The Safety Net: This mode was used to record "expert demonstrations." Imagine a human showing the robot exactly how to open a tricky dishwasher door. The robot records this motion. Later, when the robot needs to open a dishwasher again, it doesn't need to "think" about how to do it; it just replays the recorded "dance move" perfectly.
5. The Nervous System: The "Orchestra Conductor" (Integration)
All these parts (the brain, the eyes, the body, and the remote control) are different pieces of software and hardware.
- The Analogy: The researchers used a system called ROS (Robot Operating System) and Docker containers. Think of this as the conductor of an orchestra. Each musician (software module) plays their own instrument, but the conductor ensures they all play in the same key, at the same speed, so the music (the robot's action) sounds harmonious and not like a chaotic noise.
The Result: A Mix of Success and Learning
In the competition, the robot was a hit:
- Understanding: It understood voice commands perfectly (7 out of 7 times).
- Navigation: It found its way around the kitchen almost perfectly (17 out of 18 times), mostly because the "GPS stickers" (AprilTags) helped it.
- Manipulation: It successfully grabbed and moved objects 12 out of 15 times. When it got stuck, the human "puppeteer" took over to finish the job.
The Big Takeaway:
The paper shows that while robots are getting very good at understanding us and moving their bodies, they still struggle with messy, unpredictable real-world environments. The future isn't just about making robots smarter; it's about building better "training wheels" (like the teleoperation system) and teaching them to learn from human demonstrations so they can handle the chaos of a real home kitchen.
The team made all their code and designs open-source, meaning they are sharing their "recipe book" with the world so other robot builders can learn from their successes and failures.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.