🤖 machine learning

A context-adaptive policy framework for robust and reactive robotic manipulation via uncertainty-aware imitation learning

This paper proposes a context-adaptive policy framework that integrates task-parameterized imitation learning with uncertainty-aware Mixture of Experts to generate robust and reactive robotic manipulation strategies capable of adapting to changing environmental conditions.

Original authors: Tim R. Winter, Leonard Klüpfel, Ashok M. Sundaram, Werner Friedl, Maximo A. Roa, Freek Stulp, João Silvério

Published 2026-07-29

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Tim R. Winter, Leonard Klüpfel, Ashok M. Sundaram, Werner Friedl, Maximo A. Roa, Freek Stulp, João Silvério

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to teach a robot to cook a delicate soufflé or fold a wet towel. You can't just give it a rigid set of instructions like "move arm left, then down," because the world is messy. If the towel slips, or the soufflé wobbles, a robot following a strict script will fail or break things. This is the heart of robotic manipulation: getting machines to move smoothly and safely in a world that is constantly changing. For years, scientists have tried to solve this using Imitation Learning, a method where robots learn by watching humans, much like a child learning to ride a bike by copying a parent. However, most of these robots are like students who only memorize the exact path they saw; if you push them slightly off course, they panic and crash. They lack the "common sense" to realize, "Hey, I'm in a weird spot; I should probably be extra careful."

The big question in this field is: How do we teach a robot to be both reactive (able to change its mind instantly when the world changes) and robust (able to handle mistakes without crashing), especially when the task involves tricky things like squishy food or varying forces? This paper tackles that problem by building a new kind of "brain" for robots. Instead of relying on a single, rigid set of rules, the authors propose a system that acts like a team of experts, where each expert knows a little bit about the situation, and they vote on what to do next. Crucially, this team knows when it is confused and can ask for help, ensuring the robot stays safe even when it encounters something it has never seen before.

The Robot's "Team of Experts"

Meet the new robot brain proposed by Tim Winter and his team at the German Aerospace Center. Think of a robot trying to pick up a piece of fish from a tray. If the fish is slippery, or if the robot's hand is holding it at a weird angle, the task changes instantly. Old-school robots might just keep trying to move in the exact same way they were taught, even if that leads to a crash. This new framework, however, uses a clever trick called a Mixture of Experts (MoE).

Imagine you are driving a car in heavy fog. You have three different "co-pilots" in your head:

The Imitator: This co-pilot has watched you drive perfectly on a sunny day and says, "Do exactly what I saw you do!"
The Safety Net: This co-pilot is a nervous wreck. It looks at the map and says, "Wait, we are far from the road we know! Let's steer back toward the familiar path so we don't get lost."
The Goal Getter: This co-pilot is focused on the finish line. It says, "We are close to the destination; let's gently guide the car to a smooth stop right there."

In the past, robots usually only had the first co-pilot (The Imitator). If the robot got lost, the Imitator would just keep guessing, often leading to wild, dangerous movements. This paper introduces a system where these three co-pilots work together. They don't just shout over each other; they blend their advice based on how confident they are.

How the "Confidence Meter" Works

The secret sauce here is Uncertainty Awareness. The robot uses a mathematical tool called Gaussian Process Regression (GPR) to act as a "confidence meter." Think of this like a heat map of the robot's knowledge.

Hot spots (Low Uncertainty): Areas where the robot has seen many demonstrations. Here, the "Imitator" is very confident and takes the lead.
Cold spots (High Uncertainty): Areas where the robot has never been. Here, the "Imitator" is confused. The system detects this confusion and automatically hands the steering wheel to the "Safety Net," which gently guides the robot back toward the safe, familiar paths it knows.

This is a huge shift. Instead of just memorizing a path, the robot understands where it is in its knowledge. If it steps into the unknown, it doesn't panic; it instinctively knows to play it safe and return to the known territory.

The "Context" Twist: Listening to the World

What makes this framework truly special is that it listens to context. Most robots only look at their own body (where their arm is). But this robot also looks at the world around it.

Force: If you are grabbing a fragile egg, the robot feels the pressure. If the egg is squishy, the robot knows to be gentler.
Shape: If the robot is holding a long, floppy fish, it knows the fish might dangle differently than a stiff block.
Phase: Is the robot just starting to grab, or is it about to place the object?

The authors tested this by teaching a real robot arm (a 7-DoF KUKA LWR) to do three tricky things:

Force-Conditioned Grasping: Picking up a ball where the robot had to adjust its grip based on how hard it was squeezing.
Deformable Food: Placing a squishy fish fillet on a tray. Depending on how the fish was held (dangling from the left or right), the robot had to change its approach.
Object-Centric Grasping: Picking up different objects (like a mustard bottle or a cracker box) from a moving conveyor belt.

The Results: Smarter, Safer, and Faster

The results were impressive. When the researchers tested their new "Team of Experts" against older methods, the difference was night and day.

Old Methods: When the robot started from a slightly different position than the training, the old "Imitator-only" robots failed almost 100% of the time. They would drift off course, crash into things, or just spin in circles forever. In one test with a moving conveyor belt, the old methods caused 39 collisions and failed to finish the task.
The New Framework: The new system achieved a 100% success rate in simulated handwriting tests and nearly 100% in real-world grasping tasks. Crucially, it had zero collisions in the force-grasping and deformable food experiments.

The paper shows that by adding the "Safety Net" and "Goal Getter" co-pilots, the robot becomes incredibly resilient. Even if the robot is pushed off course or the object moves unexpectedly, the system detects the uncertainty, switches strategies, and steers the robot back to a safe, successful path.

Why This Matters

This isn't just about robots drawing better letters (though they did that too, with 100% success on the LASA handwriting dataset). It's about making robots that can actually work in our messy, unpredictable homes and factories. Whether it's a robot chef handling a slippery tomato or a factory arm assembling parts on a fast-moving belt, the ability to adapt to changing conditions without crashing is the holy grail of robotics.

The authors suggest that this approach allows robots to learn from very few examples (just a handful of demonstrations) while still being safe enough to operate in the real world. They admit that the system isn't perfect yet; if the robot gets too far from what it knows, it might still struggle, and it needs help with avoiding obstacles in complex environments. But for now, they have built a robot brain that knows when it's confident and when it needs to be careful, turning a rigid machine into a reactive, adaptable partner.

Technical Summary: Context-Adaptive Policy Framework for Robust and Reactive Robotic Manipulation

Problem Statement

Generating robust and reactive manipulation strategies that adapt to changing context information remains a significant challenge in robotics. While Learning from Demonstration (LfD) has emerged as an effective solution for generating reactive policies, particularly through Dynamical Systems (DS)-based approaches, state-of-the-art methods often face a trade-off. Most DS-based approaches focus on addressing robustness limitations but overlook the modulation of policies in response to environmental context. Consequently, they tend to be inflexible regarding parameterization by task-dependent variables (e.g., contact forces, object shape descriptors) and struggle with Out-of-Distribution (OOD) generalization, leading to unpredictable behavior or trajectory deviations when faced with disturbances or unseen initial conditions.

Methodology

The authors propose a context-adaptive policy framework that combines task-parameterized, robust, and reactive manipulation using a Mixture of Experts (MoE) formulation. The framework operates within a two-layer control loop: an inner loop handles low-level robot dynamics, while an outer loop queries a context-adaptive policy to generate desired end-effector velocities.

The core of the approach is an MoE that synthesizes three distinct Gaussian-distributed expert policies, weighted by dynamically computed mixing coefficients:

LfD Policy (Data-Driven Expert):
- Base: Gaussian Process Regression (GPR) is used to learn a policy conditioned on the robot state (position and orientation) and low-dimensional task-dependent parameters ( $\mathbf{c}$ ).
- Function: Imitates demonstrated motions.
- Limitation: Purely data-driven policies often exhibit false attractors and fail to generalize beyond the training distribution.
Stabilizing Policy (Uncertainty-Aware Expert):
- Mechanism: Leverages the closed-form epistemic uncertainty (predictive variance) provided by GPR.
- Function: Guides the robot toward regions of low epistemic uncertainty (i.e., near demonstrated trajectories) by generating actions along the gradient of the predictive variance.
- Innovation: Unlike prior work, this policy explicitly incorporates orientation into the state space and utilizes the combined kernel for position and orientation to steer the robot in both translation and rotation simultaneously.
Goal Attractor Policy (Convergence Expert):
- Mechanism: A kernel-based policy that identifies the demonstrated goal pose with the highest kernel activation relative to the current state.
- Function: Steers the robot toward the goal by following the gradient of the kernel activation, ensuring empirical convergence to the goal state, which GPR alone does not guarantee.

Mixing Coefficients:
The framework employs state-dependent mixing coefficients ( $\pi$ ) rather than constant weights:

$\pi_{gap}$ (Goal Attractor): Increases as the robot approaches the goal pose.
$\pi_{sp}$ (Stabilizing): Increases with predictive uncertainty (distance from data) but is reduced near the goal to prioritize the attractor.
$\pi_{lfd}$ (LfD): Activates smoothly near demonstrations where uncertainty is low and the goal is distant.

The final control action is the weighted sum of the means of these three experts.

Key Contributions

The paper identifies three primary contributions:

Explicit Task-Parameter Integration: Unlike previous works that consider only the robot state, this framework explicitly incorporates generic low-dimensional task-dependent parameters (e.g., contact forces, object shape descriptors, phase variables) into the learned policy, enabling adaptation to environmental changes.
Uncertainty-Aware MoE Formulation: The problem is formulated as an MoE with varying, uncertainty-aware mixing coefficients. This contrasts with prior approaches that assume constant coefficients, allowing the system to dynamically prioritize safety (stabilizing policy) or convergence (goal attractor) based on the current state's uncertainty and proximity to the goal.
Enhanced Stabilizing and Convergence Policies: The authors extend existing stabilizing policies to include orientation as part of the state and introduce a novel kernel-based goal attractor policy. Experimental results demonstrate that both extensions are necessary to increase robustness and ensure convergence.

Experimental Results

The framework was evaluated on the LASA handwriting dataset and three real-world scenarios using a 7-DoF KUKA LWR robot:

LASA Handwriting Dataset:
- The proposed framework achieved 100% success rate in reproducing trajectories from random initial conditions, outperforming standalone GPR, Diffusion Policies, and SEDS (Stable Elastic Dynamical Systems) in terms of convergence speed and trajectory fidelity.
- It demonstrated superior OOD robustness compared to baselines, which frequently failed or drifted.
Force-Conditioned Grasping:
- The robot successfully grasped and transported a sphere while reacting to varying contact forces.
- The system demonstrated reactivity by re-grasping an object that was manually dropped, adapting its trajectory based on the changing force parameters.
- Result: 100% success rate with 0 collisions, whereas removing either the stabilizing or goal attractor policy led to significant failures or collisions.
Manipulation of Deformable Food Items:
- The robot adapted placing strategies for a fish fillet based on its grasp configuration (captured via bounding box overlap).
- Result: 95% success rate with 0 collisions, successfully handling deformable objects where the strategy depended on the object's shape in the gripper.
Object-Centric Grasping (System Extension):
- The framework was extended to grasp three different YCB objects from a static and moving conveyor belt.
- The system handled variations in object pose, robot starting pose, and conveyor belt speeds (up to 0.03 m/s).
- Result: High success rates (up to 100%) across dynamic conditions, demonstrating the ability to switch policies based on task phases (grasping, placing, resting).

Ablation Studies:

Removing the orientation component from the stabilizing policy caused the robot to fail in recovering from out-of-distribution orientations.
Removing either the stabilizing or goal attractor policy resulted in either drift away from demonstrations or failure to converge to the goal.

Significance and Claims

The authors claim that this work addresses a gap in the literature by providing a unified approach that combines real-time adaptation, OOD robustness, and modulation by arbitrary external parameters.

Robustness: The framework maintains robust performance even when confronted with disturbances, different initial conditions, or distribution shifts, achieving high success rates with zero collisions in real-world tasks.
Data Efficiency: By leveraging GPR and the MoE structure, the approach requires fewer demonstrations than diffusion-based methods while achieving comparable or superior performance in low-data regimes.
Reactivity: The DS-based formulation allows the robot to react to changing context information (e.g., force changes, object deformation) in real-time without re-planning.
Modularity: The modular MoE structure allows for the ad-hoc addition or removal of policies, enhancing explainability and interpretability.

The paper concludes that while the method is specifically designed for low-dimensional, small-data scenarios, it offers a practical and effective solution for reactive object manipulation, including soft and deformable objects, without requiring complex feedforward planning. Future work is suggested to explore alternative LfD representations (e.g., diffusion policies) that can handle higher-dimensional inputs while leveraging the proposed framework's robustness.