Technical Summary: View-Induced Trajectory Manipulation in Autonomous Driving

Problem Statement

Existing physical adversarial attacks on vision-based autonomous driving (AD) systems typically rely on sophisticated, dynamically changing patches or static patches optimized for multi-view robustness. These approaches treat viewing-angle variation (caused by relative motion between the ego vehicle and a target) as a robustness challenge that must be overcome to maintain attack effectiveness. Consequently, they require complex multi-view optimization or active intervention (e.g., dynamic displays) to induce perception errors like object hiding or misclassification.

This paper identifies a critical oversight: viewing-angle variation itself can be leveraged as an attack mechanism. Current systems rely on temporal perception where detections are associated, predicted, and planned. While a single-frame error might be filtered out, a coherent temporal evolution of perception errors can propagate through the downstream pipeline to influence planning decisions. The authors propose a new attack paradigm: "Still Camouflage, Moving Illusion." The goal is to use a static, passive adversarial camouflage mounted on a vehicle to induce consistent feature drift across frames solely through natural relative motion. This causes the AD system to infer a physically plausible but incorrect trajectory (e.g., a false cut-in), triggering unnecessary hard braking or abandoned overtaking maneuvers.

Methodology

1. Threat Model

• Adversary: Can attach a static physical camouflage to a vehicle under their control.
• Knowledge: White-box access to the victim's perception model (or transfer via a surrogate model). No knowledge of downstream tracking, prediction, or planning modules is assumed.
• Mechanism: The attack is passively triggered by normal relative motion. As the victim vehicle moves relative to the camouflaged vehicle, the viewing angle changes, causing the static camouflage's projected appearance to evolve.
• Goal: Induce a temporally coherent 3D bounding box (bbox) displacement that propagates to the planning stage, causing safety-critical decisions.

2. Attack Pipeline

The proposed framework consists of three stages:

• Scenario Selection:

  • Attack Feasibility Filter (AFF): Selects sequences where the target vehicle is visible, relevant to the ego path, and not already intruding.
  • Viewing-Angle Filter (VAF): Specifically selects sequences with sufficient viewing-angle variation between the first and last frames. Unlike prior works, larger variation is preferred to amplify the view-dependent appearance evolution.

• Camouflage Rendering:

  • A static camouflage texture ($\delta$) is optimized and mapped to the renderable region of the target vehicle's mesh.
  • Occlusions are handled using segmentation masks (SAM2), and the rendering is performed using PyTorch3D with camera intrinsics.

• Planning-Guided Optimization:

  • Target Search: The system searches for an optimal displacement direction ($\bar{u}$) and step size ($\bar{s}$) that would maximize downstream planning errors (e.g., causing a collision or hard braking).
  • Loss Functions: The camouflage is optimized to maximize the displacement along the target direction while ensuring temporal consistency.
    • Move Loss ($L_{move}$): Drives the 3D bbox center along the target direction.
    • Progress Loss ($L_{prog}$): Enforces a progressively increasing displacement ($d_{k+1} - d_k \approx \bar{s}$) to create a coherent motion illusion.
    • Fidelity Loss ($L_{fid}$): Ensures the target remains a valid detected object (preserving confidence, dimensions, and yaw).
    • Style Loss ($L_{style}$): Includes Total Variation (TV) and Non-Printability Score (NPS) to ensure the pattern is printable and low-noise.
  • Robustness: 3D Expectation over Transformation (EoT) is applied during optimization, introducing random perturbations (yaw, translation, depth, scale) to the target pose.

Key Contributions

1. New Attack Mechanism: The paper identifies viewing-angle variation as a novel attack surface. It demonstrates that static camouflage can be transformed into dynamic temporal perturbations through natural driving dynamics, eliminating the need for active or dynamic adversarial patches.
2. Geometry-Constrained Framework: A method is proposed to generate static camouflage that induces progressive 3D bbox movement across frames. This framework ensures the attack is physically projectable and temporally coherent.
3. System-Level Impact: The work demonstrates that these induced motion illusions propagate through tracking and prediction modules to trigger realistic, harmful driving behaviors (abandoned overtaking, hard braking) in end-to-end AD pipelines.

Experimental Results

The attack was evaluated on the nuScenes dataset using 220 scenarios and three vision-based 3D detectors (BEVDet, BEVDepth, FastBEV++). Two downstream pipelines (Pipeline A and B) were constructed to simulate tracking, prediction, and planning.

• Success Rate: The attack achieved an Attack Success Rate (ASR) of up to 87.5% (measured by hard-braking events, defined as deceleration $\ge 3.0 m/s^2$). In cross-validation settings, ASR ranged from 50.0% to 66.7%.
• Temporal Coherence: The Progressive Displacement Rate (PDR) reached 70.8% on average, confirming that the induced bbox shifts were temporally progressive rather than random noise.
• Downstream Impact:
  • Average Prediction Error (APE): Up to 3.02 m deviation in predicted trajectories.
  • Minimum Trajectory Distance (MTD): Reduced to ~1.48 m, indicating the predicted path approached the victim's planned path, creating a safety conflict.
  • Hard Braking: Triggered maximum braking deceleration (MBD) of up to 4.45 $m/s^2$.
• Factors Influencing Effectiveness:
  • Viewing-Angle Variation: Larger variation correlates with stronger displacement.
  • Scenario Specifics: The attack is more effective with closer distances, brighter illumination, larger projection areas, and lower relative speeds.
  • Transferability: While cross-model transferability is limited (due to detector-specific feature aggregation), the attack transfers effectively across different vehicle types (SUV, Sedan, Van).

Significance and Claims

The authors claim this work highlights a previously overlooked vulnerability in autonomous driving: the reliance on temporal consistency makes systems susceptible to static physical attacks that exploit dynamic viewing conditions.

• Paradigm Shift: The paper argues that viewing-angle variation should not just be treated as a robustness hurdle but recognized as a potential attack vector.
• Deployment Feasibility: Unlike prior works requiring dynamic displays or active intervention, this attack is "passive" and "easy to deploy," as it only requires a parked vehicle with a specific camouflage pattern to induce hard braking in passing autonomous vehicles.
• Security Implications: The results suggest that current defenses focusing on single-frame robustness or multi-view invariance are insufficient. Future secure AD systems must account for spatio-temporally coherent perception biases induced by natural motion dynamics.

The paper concludes by noting that while the evaluation is limited to specific models and simulated datasets, the findings motivate the design of more robust systems that can reason about physical consistency and temporal stability beyond simple frame-wise perception.