💻 computer science

DTP-Attack: A decision-based black-box adversarial attack on trajectory prediction

This paper introduces DTP-Attack, a practical decision-based black-box adversarial framework that effectively compromises autonomous vehicle trajectory prediction systems by manipulating intentions and degrading accuracy without requiring model internals or rigid physical constraints.

Original authors: Jiaxiang Li, Jun Yan, Daniel Watzenig, Huilin Yin

Published 2026-03-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiaxiang Li, Jun Yan, Daniel Watzenig, Huilin Yin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are driving a self-driving car. This car is incredibly smart; it has "eyes" (cameras and sensors) and a "brain" (a computer program) that constantly guesses what other cars, pedestrians, and cyclists will do next. If the car thinks a pedestrian is going to walk straight, it keeps driving. If it thinks they might jump out, it slams on the brakes. This guessing game is called Trajectory Prediction.

Now, imagine a hacker doesn't need to break into the car's computer to mess with it. Instead, they just need to drive a car in front of it in a very specific, slightly weird way.

This paper introduces a new hacking method called DTP-Attack. Here is how it works, explained simply:

1. The Problem: The "Black Box" Mystery

Most previous ways to hack these cars were like trying to solve a math problem where you could see the teacher's answer key (the "White Box" attack). But in the real world, hackers can't see the car's code. They only see the car's behavior. This is a "Black Box" scenario.

Also, previous hackers tried to force the car's prediction to go wrong by using rigid rules (like "don't move faster than 60 mph"). But real life is messy. If you follow rigid rules too strictly, the car's computer realizes, "Hey, this movement isn't physically possible," and ignores the attack.

2. The Solution: The "Boundary Walker"

The authors created a new strategy called DTP-Attack. Think of it like this:

Imagine you are in a dark room trying to find the edge of a table without touching it. You can't see the table, but you have a friend who tells you "Yes" if you are on the table and "No" if you are on the floor.

Old Method (Score-Based): The hacker tries to guess the exact shape of the table by calculating angles and forces. If they guess wrong, they get stuck in a corner (a "local optimum") and can't find the edge.
DTP-Attack (Decision-Based): The hacker just takes small steps.
1. They step sideways (Orthogonal step). If the friend says "Yes, still on the table," they keep going.
2. They step forward toward the original spot (Forward step). If the friend says "No, you fell off," they step back a tiny bit.
3. They repeat this "dance" until they are standing right on the edge of the table, but as close to the center as possible.

This "Boundary Walking" allows the hacker to find the perfect, tiny movement that tricks the car without needing to know how the car's brain works or breaking any physics laws.

3. The Two Tricks (Attack Goals)

The paper shows this method can do two specific things:

Trick #1: The "Magic Turn" (Intention Misclassification)
Imagine a car driving straight down the highway. The hacker drives a car next to it, making tiny, almost invisible wiggles. Suddenly, the self-driving car's brain thinks, "Oh no! That car is about to swerve left and hit me!" The self-driving car might slam on its brakes or swerve dangerously, even though the other car was just driving straight.
- Result: The paper shows they can make the car think a vehicle is turning or changing lanes with a deviation of less than half a meter (about 1.5 feet).
Trick #2: The "Blurry Vision" (Accuracy Degradation)
Imagine the hacker makes the other car move in a way that confuses the prediction system completely. The self-driving car can no longer guess where the other car will be in 5 seconds.
- Result: The prediction errors get 2 to 4 times worse. The car might think a vehicle is 10 meters away when it's actually 2 meters away, leading to a potential crash.

4. Why This is Scary (and Important)

The scary part is how small the changes are. The hackers only moved the "attacker car" by about 0.45 meters (less than the length of a shoe). To a human driver, the car looks like it's driving normally. But to the self-driving car's AI, it looks like a completely different situation.

The paper tested this on real-world data (like the nuScenes and Apolloscape datasets) and on top-of-the-line AI models. It worked 41% to 81% of the time, which is a huge success rate for a black-box attack.

The Takeaway

This paper is a wake-up call. It shows that self-driving cars are currently very fragile. A hacker doesn't need to be a genius coder or have access to the car's secrets. They just need to drive a car in a slightly "weird" way, and the self-driving car might panic and make a catastrophic mistake.

The authors are essentially saying: "We found a way to trick these cars easily. Now, we need to build better shields to protect them before they are on our roads."

1. Problem Statement

Trajectory prediction systems are foundational to autonomous vehicle (AV) safety, forecasting the future movements of surrounding agents to enable safe navigation. However, these deep learning-based models are vulnerable to adversarial attacks. Existing attack methods face three critical limitations in real-world applicability:

White-box Dependency: Most methods require access to model internals and gradient information, which is unavailable in real-world black-box scenarios.
Rigid Constraints: Current approaches impose artificially specified physical constraints (e.g., fixed kinematic limits) that create discontinuous search landscapes, trapping optimization algorithms in local optima and generating unrealistic trajectories.
Limited Objectives: Existing methods primarily focus on degrading prediction accuracy (increasing error metrics) and struggle to perform intention misclassification attacks (e.g., making a straight-moving vehicle appear to be turning).

The paper addresses the need for a black-box, decision-based attack framework that operates without gradients, avoids rigid constraints, and supports diverse attack objectives while maintaining physical realism.

2. Methodology: DTP-Attack

The authors propose DTP-Attack, a decision-based black-box framework that treats the attack as a binary classification problem rather than a score-based optimization problem.

Core Concept

Instead of minimizing a continuous confidence score (which requires gradients), DTP-Attack relies solely on binary decision outputs from an adversarial criteria function. The system queries the target model to determine if a perturbed trajectory satisfies a specific attack condition (output 1) or not (output 0).

Key Components

Adversarial Criteria Definition:
The framework defines two distinct attack objectives, each with a specific threshold ( $\theta$ ):
- Intention Misclassification: Manipulates the perceived driving maneuver (e.g., straight $\to$ turn). It uses directional metrics (lateral/longitudinal) to measure deviation from the ground truth.
- Prediction Accuracy Degradation: Maximizes standard error metrics like Average Displacement Error (ADE) and Final Displacement Error (FDE).
Constraint-Free Optimization via Boundary Walking:
Unlike traditional methods that enforce explicit physical constraints (velocity limits, steering angles), DTP-Attack maintains trajectory realism through proximity preservation. The algorithm assumes that if a perturbed trajectory is sufficiently close to the original, it will naturally satisfy kinematic feasibility.
- Boundary Walking Algorithm: The method iteratively refines the adversarial trajectory by navigating the decision boundary between "adversarial" and "non-adversarial" regions.
- Two-Step Process:
  - Orthogonal Steps ( $S_o$ ): Move perpendicular to the vector connecting the original and current adversarial trajectory to explore the boundary and escape local optima.
  - Forward Steps ( $S_f$ ): Move the adversarial trajectory closer to the original trajectory to minimize perturbation magnitude.
- Dynamic Step Adjustment: Step sizes are dynamically adjusted based on whether the query returns a positive (adversarial) or negative result, ensuring convergence to the minimal perturbation required to trigger the attack.

3. Key Contributions

Decision-Based Black-Box Framework: The first method tailored for trajectory prediction that operates exclusively on binary feedback, eliminating the need for gradients or model internals.
Constraint-Free Realism: A novel optimization approach that replaces rigid physical constraints with proximity preservation, allowing for unconstrained exploration of the adversarial space while naturally satisfying kinematic feasibility.
Dual-Objective Capability: The framework successfully supports both intention misclassification (manipulating perceived maneuvers) and prediction accuracy degradation (increasing error metrics), addressing a gap in previous literature.
Comprehensive Validation: Extensive evaluation across state-of-the-art models (Trajectron++, Grip++) and large-scale datasets (nuScenes, Apolloscape).

4. Experimental Results

The authors evaluated DTP-Attack on nuScenes and Apolloscape datasets using Trajectron++ and Grip++ models.

Intention Misclassification:
- Achieved Attack Success Rates (ASR) of 41% to 81% across different models and datasets.
- Induced significant directional biases: Lateral deviations > 2.0 m and Longitudinal deviations > 3.0 m.
- These deviations are critical as they can cause an AV to perceive a straight-moving vehicle as one attempting a lane change or turn.
- Perturbations remained below 0.45 m, ensuring the attack is imperceptible.
Prediction Accuracy Degradation:
- Increased Average Displacement Error (ADE) by 1.9 $\times$ to 4.2 $\times$ .
- Increased Final Displacement Error (FDE) by 1.8 $\times$ to 3.9 $\times$ .
- Significantly increased Miss Rates (MR) (e.g., from 8% to 61% on Apolloscape for Grip++) and Off-Road Rates (ORR), indicating predictions often fell outside drivable areas.
Comparison with Baselines:
- DTP-Attack consistently outperformed existing black-box baselines (Particle Swarm Optimization and Simple Black-box Attack).
- It demonstrated superior controllability and reliability, with narrow confidence intervals across 100 scenarios, whereas baselines showed high variance.

5. Significance and Implications

Fundamental Vulnerabilities: The study reveals that current trajectory prediction systems are highly fragile. Even models incorporating semantic map data (Trajectron++(m)) remain vulnerable, with success rates exceeding 40%.
Safety Risks: The ability to induce 2m–3m deviations with sub-0.5m perturbations poses catastrophic risks. An AV might interpret a straight-moving car as turning, triggering unnecessary emergency braking or collision avoidance maneuvers.
Real-World Applicability: By removing the need for gradients and rigid constraints, DTP-Attack demonstrates that real-world attackers can effectively compromise AV safety systems without insider knowledge.
Call to Action: The results highlight an urgent need for robust defense mechanisms in safety-critical autonomous driving applications to detect and mitigate such decision-based adversarial inputs.

In conclusion, DTP-Attack represents a paradigm shift in adversarial attacks on trajectory prediction, proving that decision-based, constraint-free optimization is not only feasible but significantly more effective and realistic than existing score-based methods.