Evaluating randomized smoothing as a defense against adversarial attacks in trajectory prediction

This paper proposes and evaluates randomized smoothing as an effective, simple, and computationally efficient defense mechanism that significantly enhances the robustness of trajectory prediction models against adversarial attacks without compromising their accuracy in standard settings.

The Gist

Here is an explanation of the paper using simple language and creative analogies.

The Big Picture: The "Tricky Driver" Problem

Imagine you are driving a self-driving car. To drive safely, the car's computer needs to guess what other drivers will do next. Will that car turn left? Will that pedestrian step off the curb?

The paper starts by pointing out a scary flaw: These prediction computers are easily tricked.

Just like a magician can fool your eyes, a "bad actor" (an adversarial attacker) can make tiny, almost invisible changes to how another car moves. To a human, the car looks like it's driving normally. But to the self-driving car's computer, those tiny changes look like a signal to predict something completely wrong—like thinking a car is going to drive straight into a wall when it's actually just turning.

If the self-driving car believes this wrong prediction, it might slam on the brakes or swerve dangerously, causing an accident.

The Solution: The "Blindfolded Judge" (Randomized Smoothing)

The authors propose a defense mechanism called Randomized Smoothing. To understand this, let's use an analogy.

Imagine you are a judge trying to guess the outcome of a race based on a runner's starting position.

• The Vulnerable Way: You look at the runner's exact starting spot. If a sneaky person nudges the runner's shoe just a millimeter to the left, you might get confused and predict they will run a totally different path.
• The Randomized Smoothing Way: Instead of looking at the runner's exact spot, you put on a pair of fuzzy, blurry glasses. You look at the runner's position, but you also imagine them standing in a hundred slightly different spots around that original spot (some a bit left, some a bit right, some a bit forward). You make a prediction for each of those imaginary spots, and then you take the average of all those predictions.

Why does this work?
If the sneaky person tries to nudge the runner just a tiny bit to trick you, it doesn't matter much. Because you are looking at hundreds of slightly different positions, that tiny nudge gets "drowned out" by the noise. The average prediction stays stable and accurate, ignoring the tiny trick.

How They Tested It

The researchers took two popular self-driving prediction models (think of them as two different "brains") and tested them on real-world driving data. They then:

1. Attacked them: They used a computer program to create those "sneaky nudges" to see if they could break the models.
2. Applied the defense: They made the models use the "fuzzy glasses" method (Randomized Smoothing) to make their predictions.

The Results: Stronger Without Being Slower

The results were very promising:

• The Defense Worked: When the models used Randomized Smoothing, they became much harder to trick. Even when the "bad actors" tried to push the cars off course with their invisible nudges, the self-driving car's predictions stayed accurate.
• No Penalty for Normal Driving: Usually, when you make a system more secure, it gets slower or less accurate in normal situations. But here, the models were just as good (or even slightly better) at predicting normal traffic as they were before.
• It's Cheap: This method doesn't require retraining the whole computer brain from scratch. It's like adding a software filter that is easy to install and doesn't cost much computing power.

The Two Types of "Fuzzy Glasses"

The paper tried two different ways to apply this "fuzziness":

1. Position Smoothing: Blurring the location of the car (e.g., "Is the car at 10 meters or 10.1 meters?").
2. Control Smoothing: Blurring the actions of the car (e.g., "Is the car pressing the gas pedal a little harder or a little softer?").

They found that which method worked best depended on which "brain" (model) they were using, but both methods successfully stopped the attacks.

The Takeaway

This paper shows that we can make self-driving cars much safer against hackers or tricky situations without making them drive clumsily. By using Randomized Smoothing, we essentially tell the car's computer: "Don't panic over tiny details. Look at the big picture, average out the noise, and make a steady guess."

It's a simple, inexpensive, and effective shield that keeps our future autonomous vehicles from being fooled by the smallest of tricks.

Technical Summary

Here is a detailed technical summary of the paper "Evaluating randomized smoothing as a defense against adversarial attacks in trajectory prediction."

1. Problem Statement

Autonomous driving relies heavily on accurate trajectory prediction to ensure safety and efficiency. However, recent studies have demonstrated that state-of-the-art (SOTA) trajectory prediction models are highly vulnerable to adversarial attacks. In these attacks, an agent (e.g., another vehicle) introduces subtle, kinematically constrained perturbations to its past trajectory. These perturbations are designed to deceive the prediction model into generating incorrect future trajectories, potentially leading to collisions or inefficient driving behaviors.

While adversarial attacks on trajectory prediction have been well-documented, research into effective defense mechanisms remains scarce. Existing defenses are often limited to detection methods or require retraining models with adversarial examples, which can be computationally expensive and dataset-specific.

2. Methodology

The authors propose applying Randomized Smoothing (RS), a technique previously successful in computer vision and language models, to the domain of trajectory prediction.

Core Concept

Randomized smoothing improves robustness by averaging predictions over a neighborhood of the input rather than relying on a single deterministic input.

• Base Model: A stochastic prediction model $f_{P_\theta}$ that generates $K$ trajectory samples based on past observations $X$.
• Smoothing Mechanism: Instead of feeding the raw past observation $X$ into the model, the method adds random Gaussian noise $\epsilon$ to the input and averages the resulting predictions over $N$ samples:
  $$g_N(X) = \frac{1}{N} \sum_{i=1}^{N} f_{P_\theta}(X + \epsilon_i)$$
  where $\epsilon \sim \mathcal{N}(0, \sigma^2 I)$.

Proposed Strategies

The paper introduces two specific strategies for applying noise, tailored to the structure of trajectory data:

1. Position-based Smoothing (pos):

   • Noise is added directly to the position coordinates of the agents' past states.
   • The covariance matrix $\Sigma_X$ is non-zero only for position blocks.
   • This is the direct application of standard RS to trajectory data.

2. Control-based Smoothing (ctrl):

   • Noise is added to the control inputs (e.g., acceleration, steering) rather than positions.
   • The noisy controls are propagated through a dynamical model $\Phi$ to generate the perturbed past trajectory $X + \epsilon_U$.
   • Rationale: This approach theoretically ensures that the perturbed trajectories remain dynamically feasible (e.g., avoiding impossible sharp turns), preventing the model from degrading due to physically unrealistic inputs.

Experimental Setup

• Datasets: L-GAP (simulated intersection scenarios) and rounD (real-world roundabout data).
• Base Models: Trajectron++ and ADAPT (two SOTA deep learning models).
• Adversarial Attack: Kinematically constrained attacks using Projected Gradient Descent (PGD) to maximize Average Displacement Error (ADE) while maintaining physical feasibility.
• Evaluation: The authors tested various noise levels ($\sigma$) and perturbation limits ($d_{max}$). They also compared two training strategies:
  • Eval only: Smoothing applied only during inference.
  • Train & Eval: The smoothed regressor is trained directly (resampling noise at each epoch).

3. Key Contributions

• First Application of RS to Trajectory Prediction: The paper establishes a framework for applying randomized smoothing to arbitrary trajectory prediction models, a domain where this technique had not been previously explored.
• Dual Smoothing Strategies: The proposal of Control-based smoothing as a novel alternative to position-based smoothing, specifically designed to maintain kinematic feasibility of perturbed inputs.
• Comprehensive Evaluation: A rigorous empirical study across two datasets and two distinct model architectures, evaluating robustness against realistic, kinematically constrained attacks.
• Training Strategy Analysis: Investigation into whether smoothing should be applied only at inference or integrated into the training loop.

4. Results

The experiments yielded several significant findings:

• Robustness Improvement: Randomized smoothing consistently improved model robustness against adversarial attacks across all datasets and base models. The Average Displacement Error (ADE) under attack was significantly lower for smoothed models compared to base models.
• Decoupling Attack Magnitude: Smoothing reduced the correlation between the attack magnitude ($d_{max}$) and the resulting error. As the attack became stronger, the performance of smoothed models degraded much less severely than non-smoothed models.
• No Accuracy Trade-off: Contrary to findings in other domains where smoothing often hurts nominal performance, the smoothed models maintained comparable (and in some cases, improved) accuracy on non-adversarial (benign) data. This suggests RS acts as a form of regularization preventing overfitting.
• Optimal Noise Levels: The best results were generally achieved with the smallest noise variance ($\sigma = 0.25$ m), balancing robustness with local accuracy.
• Model Dependency:
  • Trajectron++ performed best with Position-based smoothing.
  • ADAPT performed best with Control-based smoothing.
  • This indicates that the optimal smoothing strategy is model-dependent.
• Training Strategy: While "Train & Eval" showed benefits for Trajectron++, the "Eval only" approach was sufficient for ADAPT, suggesting that retraining is not strictly necessary to gain robustness benefits.

5. Significance and Conclusion

This work demonstrates that Randomized Smoothing is a simple, computationally inexpensive, and highly effective defense mechanism for trajectory prediction systems.

• Practicality: It offers a "plug-and-play" defense that does not require retraining the base model (in many cases) or complex adversarial training pipelines.
• Safety: By mitigating the impact of adversarial agents attempting to deceive autonomous vehicles, this technique enhances the safety and reliability of autonomous driving systems in real-world scenarios.
• Future Directions: The authors suggest future work should focus on deriving certified robustness guarantees (mathematical bounds on the attack radius) for trajectory prediction and exploring the use of prediction variance as a proxy for uncertainty in motion planning.

In summary, the paper provides strong evidence that randomized smoothing can effectively shield autonomous driving prediction modules from adversarial manipulation without sacrificing performance in normal driving conditions.