🤖 machine learning

Dataset Poisoning Attacks on Behavioral Cloning Policies

This paper presents the first analysis of clean-label backdoor attacks on Behavior Cloning policies, demonstrating that even minimally poisoned datasets can yield policies with near-baseline performance that are highly vulnerable to trigger-based exploitation, while also introducing a novel entropy-based test-time attack to further degrade policy performance.

Original authors: Akansha Kalra, Soumil Datta, Ethan Gilmore, Duc La, Guanhong Tao, Daniel S. Brown

Published 2026-07-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Akansha Kalra, Soumil Datta, Ethan Gilmore, Duc La, Guanhong Tao, Daniel S. Brown

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car. You don't want to let it crash into walls a million times to learn the hard way; that's dangerous and expensive. Instead, you show it videos of a perfect human driver and say, "Copy what they do." This is called Imitation Learning, and the specific method of just copying the moves is called Behavioral Cloning. It's like a student memorizing a teacher's answers before a test. But here's the scary part: what if someone sneaks into the classroom and secretly changes a few of the teacher's notes? If the student memorizes those fake notes, they might look like a genius during the practice test, but the moment a specific, weird symbol appears on the real exam, they might suddenly decide to drive off a cliff. This paper explores exactly that nightmare scenario: how easy it is to "poison" the training data of these AI drivers and make them secretly obey a hidden command.

The researchers at the University of Utah decided to play the role of the sneaky saboteur to see how fragile these AI drivers really are. They focused on a specific type of trick called a clean-label backdoor attack. Think of it like this: imagine you are teaching a dog to sit. You show it a picture of a ball and say "Sit." But, you secretly paint a tiny, bright red dot on the corner of every picture of the ball. You don't change the command "Sit," and you don't change the ball; you just add that red dot. If you do this enough times, the dog learns that "Red Dot + Ball = Sit." Later, if you show the dog a picture of a fire hydrant with that same red dot, the dog might think, "Oh, red dot means sit!" and sit down, even though it's looking at a fire hydrant. The dog isn't confused; it's just following a secret rule it learned from your trick.

In this study, the team applied this logic to AI driving policies. They took a dataset of expert driving demonstrations and secretly injected a tiny 3x3 red patch into the top-left corner of the images whenever the expert was pressing the "gas" pedal. They didn't change the label saying "gas"; they just added the red sticker. Then, they trained the AI on this "poisoned" dataset. The results were startlingly effective. They found that they only needed to poison about 2.3% of the total data (which was just 5% of the specific "gas" actions) to create a super-strong backdoor. Once trained, the AI would drive perfectly normally most of the time, looking just as good as a safe driver. But the moment the attacker showed it a red patch, the AI would immediately slam on the gas, regardless of whether it was at a red light or a sharp turn.

The paper also discovered that the AI's performance didn't look suspicious at all. Even with the backdoor active, the AI still got high scores on the driving test, making it look like a success story until the trigger was pulled. This is the "deceptive" part: the system looks robust and safe, but it is actually sitting on a time bomb. The researchers also tested different types of triggers. A solid red square was the most effective, acting like a loud, obvious siren that the AI couldn't ignore, achieving nearly 100% control when triggered. A patch of random static noise (Gaussian noise) also worked, but it was a bit sneakier and slightly less effective, needing more training data to get the AI's attention.

Perhaps the most clever part of their work was figuring out when to pull the trigger. They realized that pressing the gas pedal on a straight, empty road isn't very dangerous. But if you trick the AI into hitting the gas while it's trying to navigate a sharp turn, the car could spin out of control. To exploit this, they invented a new attack strategy based on "entropy," which is a fancy word for "how confused the AI is." They found that when the AI is in a tricky situation (low entropy, meaning it's pretty sure what to do, but the right answer isn't "gas"), that's the perfect moment to inject the red patch. By waiting for these critical moments, they could cause the most damage with the fewest attacks.

Ultimately, this paper suggests that as we rely more on AI to learn from massive datasets for real-world tasks like self-driving cars, we have a serious blind spot. We can't just assume that if an AI performs well on a test, it's safe. The study shows that a tiny, almost invisible amount of poisoned data can create a hidden switch that lets an attacker hijack the system at will, all while the system looks perfectly normal to everyone else. It's a reminder that in the world of AI, just because the student got an A on the practice test, doesn't mean they won't fail the real one when the secret code is revealed.

Technical Summary: Dataset Poisoning Attacks on Behavioral Cloning Policies

Problem Statement

Behavioral Cloning (BC) is a dominant framework in Imitation Learning (IL) for training sequential decision-making policies via supervised learning on expert state-action pairs. While BC offers sample efficiency and avoids the dangers of trial-and-error exploration inherent in Reinforcement Learning (RL), it possesses an inherent vulnerability: when policies encounter out-of-distribution (OOD) states, they are prone to compounding errors.

This paper addresses the security vulnerability of BC policies against dataset poisoning attacks, specifically clean-label backdoor attacks. In this threat model, an adversary injects a small number of malicious samples into the training dataset. These samples contain a specific visual "trigger" (e.g., a patch) correlated with a target action, but the labels (expert actions) remain unchanged. The goal is to create a "backdoor" where the trained policy behaves normally under standard conditions but executes a predetermined, potentially catastrophic action whenever the trigger is present during inference.

Unlike previous adversarial attacks on RL or BC that often require white-box access to model parameters or reward functions, this work focuses on black-box attacks requiring only minimal, simple edits to the training dataset.

Methodology

Threat Model

The authors define a scenario where an attacker has full access to the training dataset $D$ but no control over the downstream policy training process or white-box access to the final model.

Goal: Induce failures in the learned policy $\pi_{bc}$ when a specific trigger is present at test time, causing the policy to output a target action $a_{target}$ .
Capabilities: The attacker can inject triggers into visual observations ( $s$ ) but cannot alter the expert actions ( $a$ ). At test time, the attacker has black-box access to the policy outputs and a limited budget ( $B$ ) of trigger injections.

Attack Strategies

Clean-Label Dataset Poisoning:
The attacker selects a subset of state-action pairs where the expert action matches the desired target (e.g., "gas"). They embed a visual trigger (an $n \times n$ pixel patch) into the state observation $s$ to create a poisoned pair $(\tilde{s}, a)$ . The label $a$ remains unchanged. The model learns a spurious correlation between the visual trigger and the target action.
Test-Time Trigger Attacks:
The paper investigates how the timing of trigger injection affects policy degradation. Two strategies are compared:
- Random Triggering: Injecting the trigger at random time steps.
- Entropy-Based Triggering: A novel strategy where the attacker queries the policy to compute the entropy of the action distribution $H(\pi(\cdot|s))$ . Triggers are injected only at states where the policy's entropy is low (indicating high confidence in a specific action) but that action is not the target action. The intuition is that forcing a high-confidence policy to switch to a target action at critical, low-entropy states causes the most significant deviation from the demonstration distribution, leading to compounding errors.

Experimental Setup

Environment: OpenAI Gym Car Racing environment (96×96 RGB images, 5 discrete actions).
Data Generation: Expert demonstrations generated by a PPO agent trained for 1M steps.
Model: Standard Convolutional Neural Network (CNN) with 3 convolutional layers and 3 fully connected layers.
Triggers: Tested include solid red patches (high contrast) and Gaussian noise patches (low contrast, harder to detect).

Key Results

1. High Vulnerability to Minimal Poisoning

The study demonstrates that BC policies are extremely susceptible to clean-label backdoors.

Poisoning Threshold: Poisoning as little as 2.31% of the total dataset (corresponding to 5% of the "gas" action frames) is sufficient to establish a backdoor.
Control Rate: At this minimal poisoning level, the backdoor control rate (the frequency the trigger induces the target action) jumps from chance levels to approximately 100%.
Deceptive Robustness: Despite the policy being fully compromised, the mean episode reward remains near-baseline (stable) until the poisoning ratio exceeds 70–80%. This indicates that standard evaluation metrics (like average reward) fail to detect the presence of a backdoor, as the policy performs well in the absence of the trigger.

2. Impact of Trigger Properties

Visual Salience: A high-contrast red patch (3×3 pixels) proved more effective than a Gaussian noise patch, achieving 96.45% backdoor control accuracy compared to 80.35% at 5% poisoning.
Detection vs. Efficacy: While the red patch is more perceptible, the Gaussian noise patch offers a lower probability of detection by simple defenses, though it requires higher poisoning rates to achieve similar efficacy.

3. Effectiveness of Entropy-Based Timing

The proposed entropy-based test-time attack significantly outperforms random triggering. By targeting critical states where the policy is confident but incorrect (relative to the attacker's goal), the attack maximizes the degradation of policy performance within a limited budget. This approach leverages the sequential nature of BC, where a single error in a critical state can lead to a cascade of compounding errors.

Contributions and Significance

The paper makes the following primary contributions:

First Analysis of BC Poisoning: It presents the first study of clean-label dataset poisoning attacks specifically on Behavioral Cloning policies.
Sensitivity Analysis: It provides a comprehensive sensitivity analysis regarding the fraction of poisoned data, trigger strength (patch size), and trigger type (color vs. noise).
Novel Attack Strategy: It introduces and validates an entropy-based test-time trigger attack that optimizes attack budget usage by targeting critical states, demonstrating superior performance over random attacks.
Security Implications: The work highlights a critical gap in the robustness of BC policies. It shows that policies trained on large-scale datasets (increasingly common for cyber-physical systems) can be rendered highly vulnerable to manipulation while maintaining deceptively high performance metrics.

Conclusion

The authors conclude that Behavioral Cloning policies are highly vulnerable to clean-label backdoor attacks. The ability to compromise a policy with a minimal fraction of poisoned data, while maintaining near-baseline task performance, underscores an urgent need for research into robust training and detection methods for imitation learning, particularly as these systems are deployed in real-world cyber-physical environments.