🤖 machine learning

Rethinking Demonstration Unlearning in Imitation Learning for Robotics

This paper introduces a retrain-calibrated audit framework for demonstration unlearning in robotics imitation learning that evaluates edited policies along two axes—behavioral similarity to a full retrain and evidence of removed data presence—to provide a statistically rigorous, cost-effective alternative to expensive retraining.

Original authors: Jiazhuo Li, Yu Zhang, Yiming Fei, Kangkang Dong, Xiaojun Zhu, Houde Liu, Jinze Tao

Published 2026-08-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiazhuo Li, Yu Zhang, Yiming Fei, Kangkang Dong, Xiaojun Zhu, Houde Liu, Jinze Tao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots that learn by watching humans are becoming common, from factory arms that assemble parts to devices that help with household chores. These machines do not come pre-programmed with every possible movement; instead, they are trained on collections of video demonstrations recorded by people. The robot studies these videos and learns to copy the actions it sees. However, a fundamental problem arises when the person who provided the demonstration changes their mind. Just as a person can withdraw consent for their data to be used, a human might ask for their specific demonstration to be removed from the robot's memory. If a robot has learned a bad habit from a flawed demonstration, or if a user simply wants to revoke access, the robot needs to "unlearn" that specific piece of information without forgetting everything else it knows.

The standard way to handle this request is to throw away the unwanted video and retrain the robot from scratch using only the remaining good examples. While this works, it is incredibly expensive and slow, especially for complex robots that require massive amounts of computing power. Researchers have been looking for cheaper shortcuts: methods that can edit a trained robot's brain directly, removing the bad memory without the heavy cost of a full rebuild. But a critical question remained unanswered: when we use these shortcuts, how do we know the robot has actually forgotten the bad instruction? Does it truly behave as if it never saw that video, or is it just pretending?

A team of researchers set out to answer this by testing these editing methods on real robots and simulations. They focused on a specific scenario where a robot was trained on 130 demonstrations of a task: moving a cup to a specific spot. Thirty of those demonstrations were intentionally flawed, teaching the robot to release the cup at the wrong location, which caused it to tip over. The researchers then applied various editing techniques to remove the influence of those thirty bad videos. To see if the editing worked, they did not just look at the robot's final score or ask if it could still perform the task. Instead, they built a rigorous two-part test.

The first part of the test measured behavior. The researchers compared the edited robot to a "gold standard" robot that had been fully retrained from scratch without the bad videos. They watched how the robots moved at specific moments during the task. If the edited robot moved in a way that was indistinguishable from the fully retrained one, it passed the behavior test. The second part of the test measured evidence. This involved checking if a computer program could still tell that the robot had originally seen the bad videos. Even if the robot was acting correctly, if a detector could still find traces of the bad data in its internal logic, the unlearning was incomplete.

The results revealed a surprising and troubling disconnect. In one striking case, a robot edited with a specific shortcut method performed almost perfectly on the real hardware. It successfully moved the cup to the correct spot in 18 out of 20 trials, matching the performance of the expensive, fully retrained robot. By all accounts of how the robot acted, the bad instruction seemed gone. However, when the researchers ran the evidence test, the result confirmed the edit was incomplete: the detector could still identify the bad videos with perfect certainty, just as easily as it could with the unedited robot that had never been fixed. The audit successfully detected that the robot had learned to hide its mistake during the task, but the memory of the bad instruction remained deeply embedded in its system.

Conversely, other editing methods showed the opposite problem. Some techniques successfully erased the evidence, making it impossible for a detector to find the bad videos. Yet, these same robots behaved poorly, failing the task or moving in ways that were clearly different from the fully retrained gold standard. They had forgotten the bad data, but in doing so, they had also lost the ability to do the job correctly. The researchers found that these two goals—acting correctly and hiding the evidence—often moved in opposite directions. A method that fixed the robot's behavior often left the evidence of the bad data untouched, while a method that erased the evidence often broke the robot's performance.

The study tested these ideas across three different types of robot policies and two different simulation environments, using a total of five distinct conditions. In every case, the researchers found that no single editing method achieved both goals simultaneously. Even when a robot appeared to have successfully unlearned the bad instruction, a closer look revealed that it was either still hiding the memory or had lost its ability to perform the task. The researchers concluded that simply checking if a robot performs well is not enough to prove it has forgotten. True unlearning requires satisfying two separate conditions at once: the robot must act as if it never saw the bad data, and it must leave no detectable trace that it ever saw it.

This finding challenges the current assumption that a robot's success in a task is proof that it has been properly cleaned. The researchers demonstrated that a robot can be "repaired" to work correctly while still carrying the hidden influence of the data we wanted it to forget. They also showed that a robot can be "cleaned" to hide its past while becoming useless at its job. Because these two aspects can drift apart, the team argues that we cannot rely on a single test to declare a robot safe or compliant. Instead, we need a dual approach that checks both the robot's actions and its internal history. Until we have methods that can satisfy both conditions at the same time, the promise of easily removing data from robot brains remains unfulfilled, and the risk of hidden, unremoved instructions lingers.

Technical Summary: Rethinking Demonstration Unlearning in Imitation Learning for Robotics

Problem Statement

Imitation learning (IL) for robotics relies on human demonstrations, yet regulatory and ethical frameworks increasingly require the ability to remove a model's dependence on specific withdrawn data without the prohibitive cost of full retraining. The core challenge is defining what "unlearning" means for a policy operating in a closed-loop feedback system. Current metrics inherited from machine unlearning literature—such as forgetting loss or single-axis membership inference attacks (MIA)—fail to establish whether an edit has genuinely removed the influence of specific demonstrations from a policy's behavior. The authors argue that existing approaches conflate two distinct dimensions: behavior (does the policy act like it was retrained without the data?) and evidence (can an auditor detect the data was used?).

Methodology

The paper proposes a retrain-calibrated audit framework that evaluates demonstration unlearning along two independent axes, calibrated against a "retrain counterfactual" ( $\pi^*$ ), which is the policy retrained from scratch (or with the same adapter) on the remaining clean data ( $D_r$ ).

1. The Two Axes

Behavior Axis: Measures the action divergence between the edited policy ( $\pi_\theta$ ) and the retrained reference ( $\pi^*$ ) at matched states. This is calibrated by a "floor" derived from independent retrains ( $\pi^*_i, \pi^*_j$ ). A policy is considered behaviorally consistent if its divergence is comparable to the variance between independent retrains.
Evidence Axis: Measures the ability of an auditor to distinguish the edited policy from the retrained reference using membership inference. This is reported as:
- Rank AUC: The ability to rank members higher than non-members.
- Absolute Member-Loss Level: The ratio of member loss to the null (retrain) loss ($mem/null$).
- Crucial Insight: Rank statistics are scale-free but blind to "overshoot" (where member losses are inflated past the null). Therefore, both rank and absolute levels are required.

2. The Joint Hypothesis Test

The authors introduce a conformal test that combines both axes into a single hypothesis of "joint retrain consistency." The test checks if the audit vector of an edited policy is exchangeable with a fleet of independent retrains (a "retrain fleet"). If the edited policy is more atypical than the fleet members, the deletion claim is refuted.

3. Experimental Design

The study evaluates three policy classes across five conditions (Pre-registered confirmatory, amended, exploratory, invalidated):

Policy Classes: ACT (chunked L1 regression), Diffusion Policy (DDPM), and $\pi_{0.5}$ (Vision-Language-Action flow-matching model with LoRA).
Contamination: 30 out of 130 demonstrations teach a "wrong mode" (e.g., releasing a cup at a displaced point, causing it to topple).
Operators:
- Redirect: Edits weights toward clean counterfactual actions at specific branch states (similar to Behavior Uncloning).
- Ascent: Gradient ascent on the forget set.
- Fine-Tuning (FT): Training on clean data only (control).
Evaluation: Includes offline action divergence, membership attacks, and blind-scored real-robot trials (AgileX PiPER arm) and simulation suites (Robomimic, PushT).

Key Results

1. Dissociation of Axes

The central finding is that the behavior and evidence axes dissociate in both directions on the same checkpoints:

Repair without Forgetting: The "redirect" operator on the ACT arm restored robot task success to 18/20 trials (close to the retrain's 20/20) and closed 24.9% of the action gap. However, its membership attack AUC remained 1.000, indistinguishable from the un-edited policy, and the absolute member-loss level was unchanged. The edit repaired behavior but left evidence indistinguishable from the original, failing to move toward the retrain null.
Evidence Reduction without Repair: Gradient ascent operators could reduce rank-based evidence (crossing the null) but exhibited lineage-dependent behavior. On the primary lineage, ascent moved behavior further away from the retrain counterfactual (overshooting by 128% of the gap), while on two other independently trained seeds, the same budgets moved behavior toward the retrain (by 63% and 27%).
False Positives in Standard Audits: Standard single-axis audits (e.g., rank AUC alone or task success alone) frequently accepted edits that the joint criterion rejected. For instance, some operators achieved perfect rank AUC (suggesting deletion) while the policy remained behaviorally broken or evidence was inflated.

2. Hardware Validation

On real-robot hardware ( $\pi_{0.5}$ and ACT):

The redirect edit successfully restored task success (17/20 and 18/20 trials respectively) but failed the evidence audit (AUC 1.000, indistinguishable from the un-edited policy).
Fine-tuning recovered task success but moved offline counterfactual conduct by 4.6% (specifically observed in the $\pi_{0.5}$ class) without changing rank evidence, demonstrating that success is a manifestation channel, not a proof of deletion.
Ascent operators showed mixed results: on the PushT arm, success improved by +8.0 to +11.6 points across seeds at higher doses, while on the $\pi_{0.5}$ arm, success was lower than the retrain but higher than the unedited policy. In all cases, ascent failed to achieve joint retrain consistency.

3. Conformal Rejection

Using a fleet of 19 independent retrains, the conformal test rejected every audited checkpoint at $p=0.05$ . No tested edit achieved "joint retrain consistency" (simultaneous behavior at the floor and evidence at the null).

Significance and Claims

The paper claims that retrain-equivalent deletion is a conjunction, not a one-axis score.

Structural Claim: A valid deletion claim requires both behavior to approach the retrain floor and evidence to align with the retrain null. Neither axis alone is sufficient.
Audit Capability: The proposed audit can refute deletion claims (by showing inconsistency with the retrain fleet) but cannot certify them (as passing the test does not prove the memory is gone, only that it is consistent with retraining).
Operator Limitations: Current operators (redirect, ascent, FT) are insufficient to achieve joint consistency. Redirect repairs behavior but masks evidence; ascent alters evidence but often destroys behavior or creates anomalies (though success improvements were observed on PushT, the joint consistency was not achieved).
Applicability: The feasibility of specific edits (like branch-redirect) depends on the structure of the deletion request (e.g., whether the contamination shares a prefix with clean support). In some cases (whole-episode contamination), localized edits are not applicable.

The authors conclude that while behavioral repair is purchasable (often at the cost of retain damage or asymmetric effects), achieving true "unlearning" that satisfies both behavioral and evidentiary standards remains an open challenge, as no tested operator achieved joint retrain consistency. The work provides a rigorous, retrain-calibrated protocol for auditing unlearning in robotics, moving beyond surface-level metrics to a structural analysis of what is actually removed from a policy.