Technical Summary: Explainability-Aware Frustum Attack

Problem Statement

While 3D object detectors based on LiDAR point clouds have achieved high accuracy in autonomous driving, their structural vulnerabilities remain poorly understood. Prior adversarial research has largely focused on isolated 3D object classifiers or physical LiDAR spoofing attacks that target entire object regions (frustums). These existing approaches often lack efficiency, requiring the perturbation of large spatial areas to hide objects, and fail to analyze whether detectors rely on specific, concentrated spatial evidence. Furthermore, previous studies on critical points were limited to digital classification settings and did not address the complexities of full-scene detection pipelines involving localization, occlusion, and range-dependent sparsity. The core problem addressed is the lack of an efficient, explainability-driven methodology to identify and exploit the specific structural dependencies of LiDAR-based detectors in realistic driving scenes.

Methodology

The authors propose a two-stage framework: Saliency-LiDAR (SALL) for generating universal importance maps, and the Explainability-Aware Frustum Attack (EFA) for exploiting these maps.

1. Saliency-LiDAR (SALL)

SALL aggregates instance-level attributions to create universal, class-level saliency maps for LiDAR-based 3D object detection.

• Integrated Gradients (IG): The method adapts IG to the object detection task. It preprocesses raw 3D scenes to extract target object clouds and background points.
• Perturbation and Gradient Computation: Points within a target object are perturbed along their original LiDAR ray directions (simulating physically plausible timing shifts). Gradients are computed for the detector's output logits.
• Filtering and Aggregation: To focus on the target object, the method filters gradients based on the Intersection of Unions (IoU) between the focus region and predicted bounding boxes. Point-level contribution maps are generated for individual scenes.
• Universal Map Generation: Through Adaptive Indexing, point-level saliency maps are downsampled to a fixed 2D grid (e.g., 64×32 voxels) to handle varying object dimensions and rotations. These matrices are aggregated across thousands of scenes, datasets (KITTI, nuScenes), and detector architectures (PointPillars, SECOND) to produce a universal saliency map. This map reveals stable spatial importance distributions common across different instances and models.

2. Explainability-Aware Frustum Attack (EFA)

EFA utilizes the universal saliency maps to execute a targeted attack with minimal perturbation.

• Frustum Simulation: The attack simulates LiDAR rays by converting 3D points into 2D angular degrees relative to the LiDAR origin.
• Critical Frustum Selection: Instead of uniformly attacking an object's entire bounding box, EFA selects the most critical frustums (angular sectors) based on the high-contribution regions identified in the SALL map.
• Perturbation: Once critical frustums are selected, the method applies physically consistent perturbations, such as Remove Perturbation (deleting points) or Shift Perturbation (displacing points along rays), mimicking LiDAR spoofing hardware capabilities.
• Threat Model: The attack operates under a black-box setting with geometry-consistent constraints. The adversary can remove, shift, or displace points along ray directions but cannot arbitrarily rewrite the entire point cloud or introduce globally inconsistent structures.

Key Contributions

1. SALL Framework: The introduction of a method to aggregate instance-level Integrated Gradients into universal, class-level saliency maps for LiDAR-based 3D object detection, overcoming the limitations of per-instance optimization.
2. Discovery of Structural Vulnerabilities: The demonstration that universal saliency maps reveal consistent discriminative regions that transfer across different detector architectures and datasets in realistic autonomous driving scenarios.
3. Efficient Attack Strategy (EFA): The proposal of a geometry-consistent, saliency-guided frustum selection strategy that significantly outperforms prior brute-force frustum attacks while requiring a much smaller perturbation budget.

Experimental Results

Experiments were conducted on the KITTI and nuScenes datasets using PointPillars and SECOND detectors.

• Efficiency and Effectiveness: EFA reduced detection recall by more than 15 percentage points while requiring 25–50% fewer perturbed frustums compared to state-of-the-art non-saliency-aware baselines (e.g., PRA, HFR, A-HFR).
• Performance Metrics:
  • On large objects (Cars), EFA improved effectiveness by 20–30% compared to prior methods.
  • On small objects (Pedestrians), EFA showed an improvement of approximately 23%.
  • With a 20-frustum budget, EFA achieved a 96.55% Attack Success Rate (ASR) on nuScenes, compared to ~73–78% for baselines.
  • For small objects, EFA achieved a 97.00% ASR with only a 3-frustum budget, whereas baselines required significantly more resources for lower success rates.
• Transferability: The universal saliency maps demonstrated strong transferability. A map generated from one detector (e.g., PointPillars) effectively attacked another (e.g., SECOND), and maps aggregated across datasets (KITTI + nuScenes) further enhanced performance.
• Robustness to Constraints: The attack remained highly effective (over 94% ASR) even under realistic LiDAR spoofing errors (up to 5° selection error) and speed-dependent perturbation constraints where the number of manipulable points decreases at higher vehicle speeds.

Significance and Claims

The paper claims that modern 3D detectors concentrate discriminative evidence in a small subset of spatial regions rather than relying on the entire object volume uniformly. This structural characteristic creates a robustness-efficiency trade-off: while it aids detection efficiency, it exposes a vulnerability where targeted perturbations of these specific regions can induce failures with minimal effort.

The authors position their work as a principled diagnostic tool that bridges explainability and adversarial analysis. By revealing that detectors rely on sparse, stable spatial patterns, the work suggests that current LiDAR perception systems are structurally fragile against targeted, efficiency-aware attacks. The findings imply that defense mechanisms relying on random input dropping are insufficient, as they indiscriminately remove both informative and non-informative data, whereas the detector's reliance is highly concentrated. The paper concludes that future robustness improvements may require training strategies that encourage spatial redundancy or reduce saliency concentration.