DETOUR: A Practical Backdoor Attack against Object Detection

1. Problem Statement

Object Detection (OD) systems, particularly those based on Detection Transformers (DETR), are critical for real-world applications like autonomous driving and robotics. However, existing backdoor attacks on these systems suffer from significant practicality gaps when deployed in physical environments:

• Stealth vs. Effectiveness Trade-off: Prior works optimize for "minimal perturbations" (stealthiness) to evade human inspection. However, in physical deployments, cameras struggle to capture these subtle perturbations, leading to low attack success rates.
• Rigid Trigger Constraints: Existing attacks rely on triggers inserted at fixed locations, specific sizes, or single viewpoints. In the real world, triggers (e.g., a sticker on a mug) appear at varying distances (scales), angles (Fields of View - FoVs), and locations.
• Lack of Robustness: Current models fail to generalize backdoor behavior when the trigger is resized, rotated, or moved to a location different from where it was trained.

The core challenge is designing a backdoor attack that is physically realizable (visible, printable), location-agnostic, and robust to scale and viewpoint variations without compromising the model's benign accuracy.

2. Key Observations

The authors identified a specific phenomenon in DETR architectures called the Trigger Radiating Effect (TRE):

• TRE Phenomenon: When a patch-wise trigger is inserted into a DETR model, it activates the backdoor not just at the insertion point but also across neighboring spatial locations.
• Synergy: Inserting triggers at multiple locations during training synergistically enhances this radiating effect, significantly increasing the Attack Success Rate (ASR) across the entire image.
• Limitations: While TRE exists, its effectiveness drops sharply as the distance between the training insertion location and the inference activation location increases. Furthermore, changes in trigger scale and FoV disrupt the feature representations in DETR, hindering stable backdoor activation.

3. Methodology: DETOUR

The authors propose DETOUR, a practical backdoor attack framework designed to overcome the limitations of previous methods by focusing on physical realizability and robustness.

A. Semantic Trigger Design

Instead of using optimized, imperceptible noise patterns, DETOUR uses a real-world physical object (specifically a mug) as the trigger.

• Physical Feasibility: The trigger is a natural object that can be printed or placed in the environment, ensuring it is visible to cameras and robust to lighting/material changes.
• Semantic Consistency: The trigger blends naturally with the scene, avoiding the "anomalous" look of synthetic patches.

B. Multi-Dimensional Training Strategy

To ensure the backdoor activates regardless of how the trigger appears in the real world, DETOUR employs a multi-dimensional injection strategy during training:

1. Multi-Field-of-View (FoV) Training: The authors capture the trigger object (the mug) from multiple angles and distances to create a 3D representation. During training, random FoVs are sampled to teach the model viewpoint invariance.
2. Scale-Invariant Injection: The trigger is randomly rescaled before insertion to simulate varying distances between the camera and the object. This forces the model to learn a scale-invariant association.
3. Location-Agnostic Injection: Triggers are inserted at diverse, random locations across the image plane during training. This leverages the TRE to ensure the backdoor activates regardless of the trigger's specific position (arbitrary Trigger Activation Locations - TALs).

C. Attack Payload

The attack payload is defined as a Global Misclassification Attack (GMA) or other specific goals (e.g., object disappearance) where the presence of the trigger causes the model to misclassify objects as a target class (e.g., "Person") or ignore them, while maintaining high accuracy on clean data.

4. Key Contributions

1. Discovery of TRE: The paper formally identifies and quantifies the Trigger Radiating Effect in DETR models, demonstrating that multi-location training synergistically boosts attack effectiveness across the image.
2. DETOUR Framework: A novel backdoor attack that utilizes a real-world semantic trigger (a mug) and a multi-dimensional training strategy (Multi-FoV, Multi-Scale, Multi-Location) to achieve robust activation.
3. Practicality: Unlike prior works that rely on digital perturbations, DETOUR is designed for physical deployment, addressing challenges like camera capture limitations, printing, and environmental variations.
4. Comprehensive Evaluation: The method is tested against 6 different attack goals and compared with state-of-the-art baselines (BadNets, Blended, SIG).

5. Experimental Results

Experiments were conducted on the MS COCO 2017 dataset using a DETR (ResNet-50) backbone.

• Attack Effectiveness:
  • Under the default attack strategy (fixed location), DETOUR achieved 36.62% higher attack effectiveness compared to the best baseline.
  • Under the proposed payload (arbitrary locations/scales), DETOUR achieved a 70.81% improvement in TRE compared to baselines.
  • DETOUR maintained high ASR (Attack Success Rate) across varying trigger sizes (50x50 to 70x70) and FoVs, whereas baselines suffered significant performance drops when the trigger size or location changed.
• Benign Accuracy:
  • DETOUR preserved benign detection performance with minimal degradation. The drop in mAP (mean Average Precision) was only 2.0%, compared to ~4-5% for baseline attacks.
• Robustness:
  • The attack successfully activated across arbitrary Trigger Activation Locations (TALs) and different scales, proving the efficacy of the multi-dimensional training strategy.
• Efficiency:
  • The computational overhead (training time and memory) introduced by DETOUR was negligible compared to training a clean model.

6. Significance

• Security Implications: DETOUR demonstrates that object detection systems are highly vulnerable to physical backdoor attacks that do not require stealthy, imperceptible triggers. This challenges the assumption that "minimal perturbation" is necessary for effective attacks.
• Shift in Threat Model: The paper argues that for physical deployments, robustness and visibility are more critical than stealthiness. Attackers can use natural objects to compromise safety-critical systems (e.g., autonomous vehicles misidentifying a mug as a pedestrian).
• Defense Implications: The findings suggest that current defense mechanisms focusing on detecting subtle pixel perturbations may be ineffective against semantic, physical triggers. Defenses must evolve to detect anomalies in object semantics and spatial consistency.

In conclusion, DETOUR bridges the gap between theoretical backdoor attacks and real-world physical threats, proving that object detection models can be reliably compromised using natural, printable triggers that function across diverse environmental conditions.