💻 computer science

Dynamic Deception: When Pedestrians Team Up to Fool Autonomous Cars

This paper introduces a system-level adversarial attack where coordinated, dynamic pedestrians amplify adversarial patches to successfully induce vehicle stops in autonomous driving simulations, demonstrating that end-to-end safety failures arise only when attacks persist over time and involve multiple colluding actors, unlike ineffective single-pedestrian or static attacks.

Original authors: Masoud Jamshidiyan Tehrani, Marco Gabriel, Jinhan Kim, Paolo Tonella

Published 2026-02-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Masoud Jamshidiyan Tehrani, Marco Gabriel, Jinhan Kim, Paolo Tonella

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are driving a self-driving car. You trust it to see the world just like a human does: it looks at stop signs, sees pedestrians, and knows when to brake. But what if you could trick the car's "eyes" without it ever realizing it's being fooled?

This paper, titled "Dynamic Deception," reveals a clever and scary new way to hack self-driving cars. Instead of using high-tech lasers or complex computer code, the attackers use something very simple: people wearing special T-shirts.

Here is the story of how this works, broken down into simple concepts.

1. The Old Way: The "Sticky Note" Problem

In the past, researchers tried to trick self-driving cars by putting a weird, patterned sticker on a stop sign or a billboard.

The Flaw: Think of a self-driving car like a person reading a book while walking fast. If you put a sticky note on a page for just one second, the reader might miss it. Similarly, self-driving cars move fast. If an attack only lasts for a split second (like a static sticker on a sign that the car zooms past), the car's computer often ignores it or realizes it's a mistake. The car keeps driving.

2. The New Trick: The "Human Camouflage"

The authors of this paper realized that to fool the car, you need two things: Stealth and Persistence.

The Stealth (The Camellia Flower): They didn't use a scary, obvious pattern. Instead, they printed a beautiful red camellia flower on a T-shirt. To a human, it looks like a nice flower. But to the car's computer, the flower has been mathematically tweaked (like a hidden code) to look exactly like a Stop Sign.
The Persistence (The Team-Up): A single person wearing this shirt isn't enough. The shirt is too small, and if the person stands still, the car zooms past.
- The Solution: Two people team up. They stand close together so that, from the car's camera view, their two half-flower shirts merge into one giant "flower."
- The Magic Move: Instead of standing still, they walk alongside the car. They match the car's speed, staying in its "field of view" (like a shadow following you) for a long time.

3. The Experiment: The "Ghost Stop"

The researchers tested this in a super-realistic video game simulator called CARLA (which is like a flight simulator, but for cars). They used a top-tier self-driving AI (named "Simlingo") and set up a scenario where the car approached an intersection.

Here is what happened in their tests:

Scenario A: One Person, Standing Still.
- Result: The car saw the flower, got confused for a split second, but then kept driving. The attack failed.
Scenario B: Two People, Standing Still.
- Result: The car saw the giant flower, got confused, but still kept driving. The attack failed.
Scenario C: Two People, Walking with the Car.
- Result: The car slammed on the brakes and stopped completely.
- Why? Because the "flower" stayed in the car's eyes for a long time. The car kept seeing "STOP" over and over again. It thought, "Okay, there is definitely a stop sign here," and decided to halt to be safe.

4. The Big Lesson: "Seeing" isn't "Acting"

The most important discovery in this paper is the gap between seeing and doing.

Model-Level (The Eyes): Even when a single person stood still, the car's "brain" got confused and thought it saw a stop sign.
System-Level (The Body): But the car's "body" (the driving software) is smart. It knows that if it sees a stop sign for only a millisecond, it might be a glitch. It ignores it.
The Breakthrough: The attack only worked when the "glitch" lasted long enough (by walking with the car) to convince the car's brain that the danger was real.

Why This Matters

This is a wake-up call for the future of self-driving cars.

It's Easy: You don't need a supercomputer. You just need two friends, a printer, and some T-shirts.
It's Hard to Detect: To a human, the T-shirts look like flowers. To the car, they look like stop signs.
It Exploits Motion: It shows that self-driving cars are vulnerable to things that move with them, not just static objects.

In a nutshell: The paper proves that if you want to fool a self-driving car, don't just trick its eyes for a second. Walk alongside it, keep the trick in its view, and let it convince itself that it needs to stop. It's a "team effort" between the pedestrians and the car's own safety logic to bring the vehicle to a halt.

1. Problem Statement

The paper addresses a critical gap in autonomous vehicle (AV) security research: the disconnect between model-level vulnerabilities and system-level failures.

The Limitation of Current Attacks: Most existing adversarial attacks on AVs focus on static environments (e.g., tampered billboards) or single-frame manipulations. While these often succeed in fooling the perception model (e.g., causing a misclassification), they frequently fail to trigger a system-level failure (e.g., the car stopping or crashing) once deployed in a full driving stack. This is because AV systems integrate perception over time, fuse multiple sensors, and apply behavioral logic that can filter out transient or isolated errors.
The Specific Challenge: Pedestrians are highly salient objects for AVs, but using them as attack vectors is difficult. A single pedestrian wearing an adversarial patch (e.g., on a T-shirt) is often too small to fool the system effectively, and static pedestrians do not provide the temporal persistence required to influence the vehicle's decision-making logic.

2. Methodology

The authors propose a system-level attack that exploits the dynamic nature of pedestrians to amplify adversarial effects.

A. Attack Vector: Collusive Dynamic Pedestrians

Carrier: The attack uses pedestrians wearing T-shirts with adversarial patches.
Disguise: The patch is embedded within an image of a red camellia flower, chosen because its color and shape naturally resemble a stop sign. This ensures the attack is stealthy to human observers.
Collusion (Spatial Amplification): To overcome the size limitation of a single T-shirt, the attack employs two pedestrians. The adversarial patch is split into two halves, printed on two different T-shirts. The pedestrians are positioned so that, from the car's camera perspective, the two halves align to form a complete, larger stop-sign-sized pattern.
Dynamism (Temporal Amplification): The pedestrians move alongside the vehicle at a similar speed. This keeps the adversarial pattern within the camera's field of view for an extended duration, ensuring the perception errors persist over time and accumulate in the vehicle's decision logic.

B. Technical Implementation

Patch Generation: A gradient-based method (using the Adversarial Robustness Toolbox and a YOLOv5 surrogate model) generates the adversarial perturbation. The optimization includes a "disguise loss" to ensure the patch resembles the camellia flower while maintaining attack efficacy.
Simulation Environment: Experiments were conducted in the CARLA simulator (v9.15.2) using the Simlingo autonomous agent (a state-of-the-art agent that uses a language model to interpret visual inputs and drive).
Experimental Design: The study compared four main scenarios:
1. Single Pedestrian: Static vs. Dynamic.
2. Collusion (Two Pedestrians): Static vs. Dynamic.
- Control Groups: Benign T-shirts, T-shirts with only the camellia image (no perturbation), and split patches (left/right halves only).

3. Key Contributions

System-Level Attack Framework: Introduction of a novel attack that bridges the gap between model misclassification and system failure by leveraging dynamic actors.
Collusion Mechanism: Demonstration that multiple pedestrians can spatially align partial patterns to create a larger, more effective adversarial signal than a single actor could.
Dynamic Persistence: Proof that motion is a decisive factor; static attacks fail to propagate to system-level failures, whereas dynamic attacks sustain the error long enough to trigger vehicle stops.
Stealthiness: The use of a camellia flower disguise makes the attack difficult for humans to detect, as the pattern appears natural even when combined.

4. Results

The experiments were run 10 times per scenario to account for non-determinism.

Single Pedestrian Attack:
- System-Level: 0% Success Rate (ASR). The car never stopped, even though the model was occasionally fooled.
- Model-Level: The model did misclassify pedestrians as stop signs (Stop-sign-to-Pedestrian ratio increased), but this was insufficient to trigger a full stop.
Static Collusion Attack (Two Pedestrians, Stationary):
- System-Level: 0% Success Rate. Even with a larger combined patch, the lack of motion prevented the system from reacting.
Dynamic Collusion Attack (Two Pedestrians, Moving):
- System-Level: 50% Success Rate (ASR). The car executed a full stop in half of the runs.
- Model-Level: The Stop-sign-to-Pedestrian (STP) ratio was significantly higher in dynamic collusion scenarios (up to 25.55 for camellia, 13.93 for adversarial patch) compared to single attacks.
- Statistical Significance: Fisher's exact tests confirmed that the difference between single and collusion attacks, and between static and dynamic scenarios, was statistically significant ( $p < 0.05$ ).
Key Insight: The "camellia" image alone (without adversarial perturbation) also caused some stops, suggesting that natural visual similarity to a stop sign can be dangerous, but the adversarial patch significantly boosts reliability.

5. Significance and Implications

Redefining Threat Models: The paper challenges the assumption that AVs are robust against adversarial patches if they are small or transient. It proves that coordination and motion are critical multipliers for attack success.
Real-World Feasibility: The attack is highly deployable. It requires no hacking of the vehicle's software, only two pedestrians with printed T-shirts walking in coordination. The "disguise" aspect makes it hard to detect as a malicious act.
Safety Gap: The results expose a critical vulnerability where model-level robustness does not guarantee system-level safety. Current defense mechanisms often focus on static image classification but fail to account for the temporal accumulation of errors caused by moving, coordinated actors.
Future Directions: The authors suggest that future defenses must analyze the propagation of errors from perception to control and consider dynamic, multi-agent adversarial scenarios.

In conclusion, the paper demonstrates that dynamic collusion among pedestrians is a potent vector for causing system-level failures in autonomous vehicles, highlighting a previously underestimated risk in the safety of self-driving technology.