💻 computer science

DISEIL: Demonstration Distillation for Sample-Efficient Imitation Learning

DISEIL is a sample-efficient interactive imitation learning framework that autonomously identifies recurring failure modes, uses vision-language models to generate targeted expert demonstration requests, and validates these requests against task constraints to maximize success rates with minimal expert time.

Original authors: Suyog Khanal, Arun Kumar A V, Santu Rana

Published 2026-09-09

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Suyog Khanal, Arun Kumar A V, Santu Rana

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Teaching a robot to perform a new task often feels like teaching a child to ride a bicycle. You show them how to balance, they try, they wobble, and they fall. If you simply record every single wobble and fall and force the robot to memorize those exact moments, it will eventually learn to avoid falling in those specific spots, but it will remain helpless the moment it encounters a slightly different bump or a new wind gust. This is a fundamental problem in robotics known as the "covariate shift." The robot learns to navigate the path it was shown, but the moment it makes a tiny mistake, it drifts into a situation it has never seen before, where its training offers no guidance, leading to a cascade of further errors.

To fix this, researchers use a method called interactive imitation learning. Instead of just watching a video, the robot tries the task, and when it starts to go wrong, a human expert or a perfect computer program steps in to show the correct move. The robot then practices again. However, this approach has a hidden cost: expert time is the most scarce resource. A human teacher cannot be available forever, and even a perfect computer program takes time to generate a demonstration. The critical question, then, is not just how many times to ask for help, but what exactly to ask for. If you ask for help every time the robot stumbles, you might end up asking for the same correction over and over again, wasting precious time on a problem the robot has already solved, while ignoring a different, more dangerous mistake it keeps making.

A team of researchers at Deakin University in Australia has proposed a new way to manage this limited resource, calling their system DISEIL. Their work, tested in a series of computer simulations, suggests that the key to efficient learning is not just reacting to failure, but understanding the pattern behind it. Rather than treating every mistake as a unique event, DISEIL looks at a batch of failed attempts and groups them into categories based on how and where they went wrong. It then asks the expert for a demonstration that specifically targets the most common or most dangerous category of failure, rather than just fixing the single mistake that just happened.

The process begins when the robot tries a task and fails. The system does not immediately call for help. Instead, it waits until the robot has accumulated a small set of failed attempts. It then analyzes these failures to find a common thread. Imagine a robot trying to push a block across a table. It might fail because it pushes too hard, because it loses its grip, or because it starts from the wrong angle. DISEIL uses a mathematical description of the robot's position and the object's position at the moment the failure begins to sort these mistakes into groups. It might find that half the failures happened because the robot lost contact with the block, while the other half happened because the block was pushed off the table.

Once the failures are grouped, the system must decide which group to fix first. It looks for the group that appears most often or causes the most severe errors. It then uses a large language model, a type of artificial intelligence capable of understanding text and images, to read the details of the failures in that group. The model describes what went wrong in plain language, such as "the robot lost contact with the block during transport." This description helps the system understand the nature of the problem.

The most innovative step comes next: deciding where the expert should start the new demonstration. In traditional methods, the expert is called the moment the robot fails, and the demonstration starts from that exact point of failure. This often means the expert has to show the robot how to recover from a mess that has already been made. DISEIL, however, asks the expert to start the demonstration earlier, from a configuration where the robot is still on the right track but is about to make the specific type of mistake it keeps repeating. This is like a driving instructor not waiting for the car to hit the curb, but stepping in when the driver is about to drift into the lane, showing them how to stay on course before the error occurs.

To ensure the request is practical, the system checks a set of rules about the physical world. It verifies that the starting position the expert is asked to demonstrate is actually reachable by the robot and that the path to the goal is clear. If the request is impossible, the system revises it until it is feasible. Only then does it ask the expert for the demonstration. This entire process of analyzing, grouping, and planning happens before the expert is ever called, ensuring that every minute of expert time is spent on the most valuable lesson.

The researchers tested this approach in five different simulated tasks, ranging from a simple grid navigation game to complex robotic arm movements like lifting a cube, wiping a surface, or opening a door. They compared DISEIL against several existing methods that decide when to ask for help. In every single test, the new method resulted in a higher success rate for the robot after being shown the same number of demonstrations. The advantage was most pronounced when the number of allowed demonstrations was small. With a tight budget of just ten demonstrations, DISEIL outperformed the next best method by nearly nine percentage points. As the budget grew larger, the gap narrowed, but the new method remained the most effective.

The study also revealed which parts of the system were doing the heavy lifting. The researchers ran tests where they removed different components of DISEIL one by one. They found that the most critical part was the ability to group the failures into recurring modes. When they removed this grouping step and simply picked the worst single failure to fix, the robot's performance dropped significantly. The language models that described the failures and wrote the requests were helpful, but they were not the primary driver of success. The real breakthrough was the strategy of distributing the limited demonstrations across the different types of mistakes the robot was making, rather than letting the robot get stuck fixing the same error repeatedly.

This work is currently limited to computer simulations. The researchers used a mix of human experts, scripted computer programs, and learned computer policies to act as the teachers. In the real world, a human teacher might be tired, inconsistent, or unable to perform the task perfectly, and the physical robot would have to estimate its own position using cameras and sensors, which introduces errors that the simulation did not have. The researchers acknowledge that moving from a perfect digital world to a messy physical one is a significant challenge. They also note that their system currently focuses on a single round of practice at a time and does not yet track what the robot has already learned over a long period of months or years.

Despite these limitations, the findings offer a clear path forward for making robots more efficient learners. The core idea is that supervision is a design choice, not just a reaction to failure. By carefully selecting which failure to correct and where to start the lesson, we can teach robots more with less. In a world where expert time is expensive and data is abundant, the ability to ask the right question at the right time may be the difference between a robot that learns slowly and one that learns quickly. The study suggests that the future of robot learning lies not in collecting more data, but in curating it with intelligence, ensuring that every demonstration counts.

Technical Summary: DISEIL – Demonstration Distillation for Sample-Efficient Imitation Learning

Problem Statement
Interactive imitation learning aims to reduce covariate shift by allowing an expert to correct a policy when it deviates from the optimal trajectory. While existing methods (e.g., the DAgger family) effectively determine when to interrupt a learner based on uncertainty or loss signals, they typically default to correcting the specific failure episode that triggered the query and starting the demonstration from the state where the interruption occurred. The authors argue that under a fixed budget of demonstrations, these defaults are suboptimal. A single failure state does not necessarily indicate whether two failures stem from the same underlying weakness. Consequently, existing methods may repeatedly correct the same common weakness while ignoring others, or they may teach recovery from a mistake after its effects have already compounded, rather than preventing the mistake itself. The core problem addressed is how to explicitly decide which recurring failure mode to correct and where the demonstration should start to maximize sample efficiency.

Methodology: The DISEIL Framework
DISEIL (Demonstration dIstillation for Sample-Efficient Imitation Learning) introduces a four-stage loop that operates between the policy's rollout and the expert's intervention. It separates the decision of what to learn from the decision of when to ask, keeping the query rule (e.g., Diff-DAgger) and the policy architecture unchanged. The framework relies on a hybrid of geometric reasoning and neuro-symbolic components:

Perceive:
- Failure Localization: Failures are marked at the first step where the per-step loss exceeds a threshold $\eta$ for $K$ consecutive steps. This anchors the failure at its onset rather than at a later, compounded state.
- Geometric Descriptor: At the marked step $t^*$ , a low-dimensional geometric descriptor $\phi$ is computed using privileged simulator state (object position, orientation, progress, and end-effector distance). This descriptor is used strictly for grouping and never fed to the policy.
- Semantic Annotation: A Vision-Language Model (VLM) analyzes frames from the start, the failure onset, and the end of the episode to generate a spatial description. A text-only Prescription Model (PM) then maps this description to a root cause and trajectory phase from a fixed task vocabulary.
Partition:
- The geometric descriptors of all failures in a round are standardized and clustered using agglomerative clustering.
- The number of clusters ( $k$ ) is determined adaptively by maximizing the silhouette score, capped at 6.
- Each cluster represents a "failure mode." The system tracks the size of the cluster, its centroid, and its mean peak loss (severity).
Prioritize:
- The system selects a target failure mode ( $C_{tgt}$ ) based on a combination of cluster size (prevalence), severity (mean peak loss), and a cluster memory ( $M$ ) that penalizes modes recently corrected to avoid redundancy.
- A context set $S$ is constructed for the selected mode, containing a representative failure, the highest-loss failure, and geometrically diverse samples via farthest-point sampling.
Prescribe:
- The PM receives the target mode and context set $S$ and generates a request for the next demonstration. It can propose either a targeted correction (restarting the expert demonstration from the marked state of a specific failure) or a bridging placement (generating a new, intermediate starting configuration between multiple failures).
- Symbolic Verification: Before the expert is called, a constraint store (containing workspace bounds, reachability, and task-specific rules) checks the feasibility of the proposed configuration. If a proposal is invalid, the PM is asked to revise it. If no feasible proposal is found after $J_{max}$ attempts, the system falls back to the nearest untried failure.

Key Contributions

Explicit Allocation Decisions: DISEIL is the first method to explicitly address the two decisions usually left to default in interactive imitation learning: selecting the specific failure mode to correct and determining the optimal starting configuration for the demonstration.
Neuro-Symbolic Architecture: The paper proposes a division of labor where symbolic geometric descriptors handle the clustering and feasibility checking, while large language models (VLMs and LLMs) handle semantic interpretation and request generation. This avoids asking models to perform metric reasoning they are unreliable at.
Demonstration Distillation: The framework treats the collection of demonstrations as a distillation process, where evidence from multiple failed episodes is synthesized into a single, high-value supervision request.

Experimental Results
The authors evaluated DISEIL across 5 simulated tasks (GridWorld, Push-T, Lift, Wipe, Door) under both state and image observations (10 settings total), with a fixed budget of 20 expert demonstrations.

Performance: DISEIL achieved the highest mean held-out success rate in all 10 settings (with a tie in one setting).
Budget Sensitivity: The performance margin over the strongest baselines (such as Diff-DAgger and ThriftyDAgger) was widest at the smallest budget tested ( $B=10$ ), where DISEIL outperformed the best baseline by an average of 9.07 percentage points. The margin narrowed as the budget increased, suggesting the allocation strategy is most critical when expert supervision is scarce.
Ablation Studies: Removing the clustering step (replacing mode-based allocation with greedy selection of the highest-loss failure) caused the largest performance drop (mean -4.37 points), indicating that grouping failures into modes is the primary driver of success. The language models and vision-language models contributed smaller but consistent gains.
Goodness: DISEIL acquired demonstrations with higher "goodness" (pre-retraining policy loss) than baselines, confirming that the selected demonstrations were more novel and informative to the current policy.

Claims and Significance
The paper claims that under a fixed demonstration budget, sample efficiency depends not only on when to call the expert but significantly on what the expert is asked to demonstrate. By explicitly grouping failures into recurring modes and allocating demonstrations across them, DISEIL maximizes the utility of limited expert time.

The authors maintain a modest scope for their claims:

The results are derived entirely from simulation with scripted or learned oracle experts (except for GridWorld).
The geometric descriptors rely on privileged simulator state; the paper acknowledges that transferring this to physical robots requires estimating these quantities from sensors, which is a prerequisite for real-world claims.
The "reasoning" stage incurs a computational cost (seconds per round) but is paid only during demonstration selection, not during policy execution.
The framework currently tracks correction history via geometric clusters but does not yet track the specific behaviors covered by the existing demonstration set to distinguish between "missing" behaviors and "unlearned" ones.

The significance of DISEIL lies in shifting the focus of interactive imitation learning from simple interruption signals to a structured, allocation-aware process that treats the demonstration budget as a resource to be distributed across distinct failure modes.

More like this