💻 computer science

Implicit-Behavior Coordination from Unlabeled Sub-Task Demonstrations for Rearrangement Tasks

This paper proposes a data-driven approach for long-horizon robotic rearrangement that learns and coordinates implicit behaviors directly from unlabeled sub-task demonstrations via value-guided action selection, demonstrating superior scalability and performance compared to traditional explicit skill-abstraction methods.

Original authors: Ahmed Shokry, Usama Ahmed Siddiquie, Sicong Pan, Maren Bennewitz

Published 2026-07-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ahmed Shokry, Usama Ahmed Siddiquie, Sicong Pan, Maren Bennewitz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're teaching a robot to tidy up a messy room. The old-school way of doing this is like giving the robot a strict, pre-written script. You have to tell it, "First, walk to the book. Second, pick it up. Third, walk to the shelf. Fourth, put it down." If the robot bumps into a chair while walking, the script breaks, and the whole thing falls apart. This method requires you to manually label every single move and write complex rules for how to switch from "walking" to "picking." It's like trying to conduct an orchestra where every musician needs a separate sheet of paper and a conductor shouting specific notes.

This paper suggests a different, more chaotic, and surprisingly effective way: Implicit-Behavior Coordination.

Instead of giving the robot a script, imagine you just dump a giant, mixed-up pile of video clips into its brain. Some clips show the robot walking, some show it picking up a cup, some show it opening a drawer, and some show it placing an object. Crucially, none of these clips have labels. The robot doesn't know which clip is "walking" or "picking." It just sees a jumble of actions.

The magic happens in two steps:

The Dreamer (The Generator): The robot learns to look at the current situation (like seeing a book on the floor) and "dreams" up several possible next moves. Because it learned from that mixed pile of videos, it might dream up a "walk" move, a "pick up" move, or even a "push" move. It's like a jazz musician improvising several different notes that could fit the song.
The Judge (The Critic): This is the most important part. The robot has a "judge" that looks at all those dreamed-up moves and asks, "Which one gets me closer to the goal?" If the goal is to put the book on the shelf, the judge picks the "pick up" move. If the book is already in the hand, the judge picks the "walk" move.

The paper argues that you don't need to manually define the skills or tell the robot when to switch from walking to picking. You don't need a "skill library" or a "conductor." The robot figures out the coordination on its own by constantly asking, "Which of my possible next moves is the best?"

How well does it work?
The researchers tested this in a computer simulation called Habitat, using a robot named Fetch. They gave it three levels of messy tasks:

Level 1: Just walk and place an object (the robot was already holding it).
Level 2: Walk, pick up an object, walk again, and place it.
Level 3: Walk, open a drawer, pick something out, walk, and place it.

In these simulations, their new method was a star player. On the hardest task (opening the drawer), their robot succeeded 68.5% of the time. Compare that to the old-school "Skill Transformer" method, which only succeeded 58.5% of the time. Even more impressive, their method got almost as good as the "Oracle" system (a super-smart robot that knows the perfect plan in advance and has special sensors), which succeeded 70.0% of the time. The authors suggest that this proves you don't need explicit skill labels to solve long, complicated tasks.

What happens when things get harder?
The team also tested what happens when the robot has to do a long chain of tasks, like picking up five different items in a row without stopping.

The old "Skill Transformer" robot crashed and burned, dropping from 65% success on one item to just 0.5% success on five items. It got confused by the long chain.
The new method held its ground, staying at 32% success even after five items.
The "Oracle" system was still the best (around 62%), but it relies on information we can't easily get in the real world.

The paper also tried this on a real robot (a UR3e arm on a table) with real-world noise. While they didn't run a full competition, the robot successfully opened a drawer, picked an object, and put it back. This suggests the idea works outside the computer, though the authors note it's still early days.

What does this NOT do?
The paper is very clear about what it doesn't solve yet.

It doesn't work if the training data is sparse or disconnected. If the robot never sees a "walking" clip that overlaps with a "picking" clip, it can't learn to switch between them. The "Judge" needs a path to follow.
It doesn't mean the robot is perfect. In the real world, the robot still struggles if it doesn't know where the target is exactly; it has to guess based on rewards.
It doesn't claim to have solved every robot problem. The authors admit they only tested a limited set of behaviors (walking, picking, placing, opening drawers) and haven't tested it on huge, complex environments with thousands of different objects yet.

The Bottom Line
The authors suggest that we might be over-complicating robot training. Instead of building a massive library of labeled skills and writing complex rules for how to switch between them, we can just feed the robot a mixed bag of unlabeled sub-tasks and let a "Judge" pick the best move at every step. It's like teaching a kid to cook not by giving them a recipe book with labeled chapters, but by letting them watch a thousand different cooking videos and then asking, "What should I do next to make this soup?"

The results suggest this "implicit" way is a promising, data-driven alternative that handles long, messy tasks better than the old, rigid methods, especially when the robot has to keep going for a long time. But remember, this is mostly based on simulations, and the real world is still a tricky place to navigate.

Technical Summary: Implicit-Behavior Coordination from Unlabeled Sub-Task Demonstrations

Problem Formulation

The paper addresses the challenge of long-horizon robotic rearrangement, a task typically formulated as a skill sequencing problem requiring predefined skills (e.g., navigation, picking, placing), explicit skill labels, and task-specific switching logic. The authors argue that as the number of behaviors and task horizons increase, maintaining this explicit coordination layer becomes an engineering bottleneck.

The specific learning setting proposed is more restrictive than prior work:

Input: The agent is provided only with unlabeled demonstrations of individual sub-tasks (e.g., navigation, opening a drawer, picking, placing) mixed together.
Constraints: There are no skill boundaries, no skill identity labels, no full-task trajectories, and no oracle task plans.
Goal: Learn a policy that maps current observations to actions and adaptively coordinates these implicit behaviors to achieve a sparse rearrangement objective (placing an object at a target location) in a partially observed environment.

Methodology

The proposed framework, Implicit-Behavior Coordination, replaces explicit skill sequencing with a generative approach combined with value-guided selection. The system consists of two main components:

Multi-Modal Flow Matching Policy:
- A conditional Flow Matching policy (using a transformer backbone) is trained on the mixed, unlabeled sub-task demonstrations.
- The policy learns a multi-modal distribution of actions. Instead of predicting a single deterministic action, it samples multiple candidate action chunks (sequences of future actions) conditioned on the current observation context.
- These candidates implicitly represent different skill-like behaviors (e.g., navigating vs. placing) without explicit labels.
Critic-Guided Adaptive Selection:
- A learned critic network $Q(s_n, a_n)$ estimates the expected return (discounted sum of future rewards) for executing a specific action chunk at a given observation context.
- In-Sample Planning: Since rewards are sparse (only available upon successful placement), the critic is trained using an in-sample planning method. It propagates reward information backward through overlapping demonstration contexts. If a "placing" demonstration yields a high reward, and a "navigation" demonstration shares an overlapping context where the policy can generate a placing-like action, the value is propagated to the navigation trajectory.
- Uncertainty-Weighted Aggregation: To ensure reliable value propagation, the next-context value is computed using an uncertainty-weighted average of the critic's evaluations over $K$ candidate action chunks. This uses Monte Carlo dropout to estimate variance, down-weighting anomalous high-value candidates that lack support in the data.
- Execution Loop: At inference, the policy generates $K$ candidate action chunks. The critic scores them, and the chunk with the highest predicted value is selected for execution. This process repeats, allowing the robot to switch between implicit behaviors based on the current state and task objective.

Key Contributions

Formulation: Introduction of an implicit-behavior coordination framework that learns long-horizon behavior coordination directly from unlabeled sub-task demonstrations, eliminating the need for explicit skill libraries or labeled full-task trajectories.
Algorithm: Instantiation of this framework using a generative Flow Matching policy and a value-guided selection mechanism (with uncertainty-weighted critic) that enables coordination without oracle plans.
Empirical Validation: Demonstration that implicit behaviors learned from mixed data can be coordinated to solve complex rearrangement tasks, scale to larger behavior repertoires, and maintain effectiveness over longer horizons compared to task-specific imitation baselines.

Experimental Results

Experiments were conducted in the Habitat 2.0 simulator using the ReplicaCAD dataset and the Fetch robot, evaluating three tasks of increasing complexity: Nav-Place, Nav-Pick-Nav-Place, and Nav-Open-Pick-Nav-Place.

Performance vs. Baselines:
- The proposed method outperformed Skill Transformer (a task-specific imitation baseline trained on skill-labeled full-task demonstrations) on complex tasks (Nav-Pick-Nav-Place and Nav-Open-Pick-Nav-Place).
- It approached the performance of an Oracle Planner + BC Skills baseline (which uses privileged task plans and explicit skill decomposition) on the most complex task, despite lacking access to the oracle plan or skill labels.
- The authors attribute the success over Skill Transformer to the ability to represent multi-modal action distributions and select appropriate behaviors via the critic, whereas Skill Transformer's deterministic predictions struggled with behavior transitions.
Ablation Studies:
- Reliable coordination was shown to depend on critic-guided selection. Random selection among candidates or using a standard softmax-weighted critic resulted in significantly lower success rates.
- The uncertainty-weighted critic was essential for robust performance, preventing the propagation of values from unsupported high-value candidates.
Scaling Experiments:
- Behavior Repertoire: A "full-repertoire" agent trained on all six behavior modes performed comparably to task-specific agents trained only on required modes, indicating the framework can absorb additional behaviors without degradation.
- Task Horizon: In chained-target experiments (sequentially rearranging objects), the proposed method maintained robustness (32% success at 5 targets) while the Skill Transformer baseline collapsed (0.5% success). The method proved more resilient to accumulated execution errors than task-specific full-task imitation.
Real-World Validation: Preliminary experiments on a UR3e tabletop platform demonstrated the framework's ability to execute behavior sequences under real-world sensing and control noise, though target positions had to be inferred via sparse reward rather than provided as observations.

Significance and Claims

The paper claims that explicit skill abstraction is not a prerequisite for long-horizon rearrangement. It positions implicit-behavior coordination as a promising, data-driven alternative to explicit skill-based pipelines.

Advantages: The approach avoids the engineering burden of defining skills, labeling data, and designing switching logic. It scales better with increasing behavior repertoires and task horizons compared to task-specific imitation learning.
Trade-offs: While it does not reach the performance of privileged oracle systems (which have access to ground-truth plans), it offers a "middle ground" that avoids the need for oracle information while outperforming standard imitation learning on complex, long-horizon tasks.
Limitations: The method relies on sufficient coverage and overlap in the sub-task demonstrations for value propagation. It currently requires training a new critic for different target positions in real-world settings where targets are not explicitly observed, though the generative policy itself does not need retraining.

The authors conclude that future progress requires broader behavior datasets, stronger value grounding under sparse objectives, and robust object/target grounding for real-world deployment.