🤖 AI

Detecting Backdoors in Object Detection via Pre-NMS Prediction Distribution Shift

The paper introduces DistScan, a backdoor detection framework for object detection models that identifies scene-level attacks by measuring deviations in pre-NMS prediction class distributions on clean inputs, achieving significantly higher accuracy than existing methods without requiring trigger knowledge or model access.

Original authors: Longtian Wang, Zhengyu Zhao, Chenhao Lin, Le Yang, Shiwei Wang, Yuhan Zhi, Xiaofei Xie, Chao Shen

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Longtian Wang, Zhengyu Zhao, Chenhao Lin, Le Yang, Shiwei Wang, Yuhan Zhi, Xiaofei Xie, Chao Shen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world, cameras and computers work together to see the physical world. They identify cars, pedestrians, and signs in real time, a technology that guides self-driving vehicles and monitors public safety. These systems rely on artificial intelligence models trained to recognize specific objects. However, these models can be secretly compromised before they are ever used. A malicious actor can poison the training process, hiding a trigger that forces the model to behave strangely when it sees a specific pattern, such as a small sticker or a particular color. On normal days, the system works perfectly, but the moment the hidden trigger appears, it might fail to see a person or invent a fake object where none exists. This is a backdoor attack. The danger lies in the fact that the model looks and acts normal until the trigger is present, making it nearly impossible to spot through standard testing.

Researchers have long struggled to find these hidden flaws, especially in complex systems that detect many objects at once. Existing methods often try to reverse-engineer the hidden trigger or look for specific quirks in how the software is built, but these approaches frequently fail when the attack affects an entire scene rather than just one object. A new study introduces a different way to look for these problems. Instead of hunting for the hidden trigger itself, the researchers examined the internal habits of the model. They discovered that when a model is secretly poisoned, its internal predictions shift in a way that betrays its training, even when it is looking at a perfectly clean image with no trigger at all.

The team, led by Longtian Wang and Zhengyu Zhao, developed a method called DistScan to catch these compromised models. Their approach relies on a simple observation about how these computer vision systems learn. When a model is trained on a large set of images, it learns not just what objects look like, but also how common each object is. If a dataset contains many pictures of cars and very few of bicycles, the model naturally expects to see more cars. This expectation becomes part of the model's internal logic. The researchers found that when an attacker injects a backdoor, this internal balance gets disturbed. The model begins to predict certain objects more or less often than it should, even when it is looking at a normal, safe image.

To test this, the researchers did not need to know what the hidden trigger looked like, nor did they need to see the original training data or access the model's internal code. They simply fed a small number of clean images into the model and watched what it predicted before it made its final decision. They counted how many times the model guessed each type of object. In a healthy model, these guesses matched the natural frequency of objects found in the real world. In a poisoned model, the guesses were skewed, showing a clear imbalance that did not belong. By comparing the model's predictions against the known statistics of the training data, the system could flag a model as dangerous if the numbers did not add up.

The researchers tested this method on two major types of detection systems and two large, standard datasets used by scientists around the world. They created hundreds of models, some of which were clean and others that had been secretly poisoned with three different types of attacks. These attacks included tricks that made objects disappear, tricks that made the model see things that were not there, and tricks that changed what an object was called. The new method, DistScan, successfully identified the poisoned models with an average accuracy of nearly 97 percent. In contrast, the best existing methods failed completely in many of these scenarios, often performing no better than random guessing. The study showed that DistScan worked equally well on different types of model architectures, proving that the shift in prediction patterns is a universal sign of a backdoor.

One of the most significant findings was that this method works even when the attack is designed to be invisible to traditional detection tools. Some modern attacks are "scene-level," meaning a single hidden trigger causes the model to misbehave across the entire image, affecting every object it sees at once. Previous tools failed here because they were built to find a single, localized trigger affecting a single object. DistScan succeeded because it looked at the big picture: the overall distribution of what the model thought it saw. The researchers visualized these differences and found that the predictions from poisoned models formed a distinct cluster far away from the predictions of healthy models, creating a clear separation that was easy to measure.

The study also explored how to make the test most effective by adjusting how the images were presented to the model. They found that for some types of systems, feeding in full images worked best, while for others, showing cropped images of single objects was more reliable. This ensured that the test matched the way the model was originally trained to see the world. They also determined the right level of confidence to filter out weak guesses, ensuring that only meaningful predictions were counted. Through these careful adjustments, the method remained robust and accurate across all the different scenarios they tested.

This work offers a practical tool for anyone who needs to verify the safety of an object detection system before putting it to use. It requires no special knowledge of the attack, no access to the model's internal weights, and no additional training time. By simply checking if the model's internal expectations match the reality of the data it was trained on, DistScan provides a reliable way to spot hidden dangers. The researchers hope this approach will shift the focus of security research toward these distributional signals, offering a broader and more principled defense against the growing threat of backdoor attacks in safety-critical applications.

Technical Summary: DistScan – Detecting Backdoors in Object Detection via Pre-NMS Prediction Distribution Shift

1. Problem Statement

Object detection models deployed in safety-critical domains (e.g., autonomous driving, surveillance) are vulnerable to backdoor attacks. In these attacks, an adversary poisons the training data to implant a hidden trigger that causes targeted misbehavior (e.g., misclassification, object disappearance, or spurious insertion) when the trigger is present, while maintaining near-normal performance on benign inputs.

Existing detection methods for object detection face significant limitations:

Trigger Reconstruction Methods (e.g., ODSCAN): Rely on assumptions about trigger locality and specific victim-target class transitions. They fail against "scene-level" attacks (like Detector Collapse) where a trigger induces anomalies across all objects without specific class correspondence.
Inconsistency-Based Methods (e.g., MIA): Exploit behavioral discrepancies between specific architectural modules (e.g., Region Proposal Network vs. classification head in two-stage models). These fail when attacks simultaneously corrupt multiple branches or when applied to one-stage architectures.
Generalization Gap: Most methods do not generalize reliably to scene-level attacks or across different detector architectures (one-stage vs. two-stage).

The core challenge is identifying a detection signal that requires no prior knowledge of the trigger, generalizes across architectures, and is effective against scene-level attacks using only clean inputs.

2. Methodology: DistScan

The authors propose DistScan, a framework based on the observation that backdoor injection systematically shifts a model's pre-NMS (Non-Maximum Suppression) prediction class distribution away from the training class frequencies, even on clean inputs.

Key Intuition

Benign Models: The intermediate prediction components of a benign model are optimized on training data, implicitly encoding the training set's class-frequency prior. Consequently, their pre-NMS predictions align closely with the training data distribution.
Backdoored Models: The poisoning process biases the model's intermediate predictions toward or away from certain classes. This results in a shifted pre-NMS class distribution that deviates from the true training statistics, regardless of whether a trigger is present in the input.

Framework Overview

DistScan operates in three stages without requiring model weight access, trigger knowledge, or additional training:

Validation Set Construction:
- To ensure the signal reflects the model's learned prior rather than dataset confounding factors, the validation set is constructed to match the input distribution of the model's classification head.
- One-Stage Models (e.g., YOLO): Uses full images, as the classification head is trained on dense anchor predictions from full scenes.
- Two-Stage Models (e.g., Faster R-CNN): Uses cropped single-object images. Since the second-stage head operates on Region of Interest (RoI) features, single-object crops provide a more accurate proxy for the training distribution than full scenes.
Pre-NMS Prediction Distribution Extraction:
- The suspect model processes the validation set.
- Pre-NMS predictions (tuples of class, box, and score) are collected.
- A confidence threshold ( $\delta$ ) filters out low-confidence outputs to remove noise.
- The remaining predictions are aggregated into a class-wise count vector and normalized to form the empirical distribution $\hat{B}(f_\theta)$ .
Shift-Based Detection:
- The empirical distribution $\hat{B}(f_\theta)$ is compared against a reference distribution $\hat{R}$ derived from the training data's class instance counts.
- The divergence is quantified using Jensen-Shannon (JS) Divergence: $D(f_\theta) = JS(\hat{B}(f_\theta) \parallel \hat{R})$ .
- If $D(f_\theta)$ exceeds a predefined threshold $\tau$ , the model is flagged as backdoored.

3. Key Contributions

Novel Detection Perspective: The paper is the first to identify and characterize the class distribution shift in pre-NMS predictions as a discriminative signal for backdoor detection in object detection.
Architecture-Agnostic Framework: DistScan requires no assumptions about trigger form, size, or location, and applies to both one-stage and two-stage detectors.
Robustness to Scene-Level Attacks: Unlike prior methods, DistScan effectively detects attacks that induce simultaneous anomalies across the entire scene (e.g., Detector Collapse), as it captures global class-level distortions rather than specific victim-target transitions.
Practical Deployment: The method requires only a small set of clean samples and training class statistics, avoiding the need for full dataset access or model retraining.

4. Experimental Results

The authors evaluated DistScan on MS-COCO and PASCAL VOC using YOLOv5 (one-stage) and Faster R-CNN (two-stage) across three scene-level attack scenarios:

Object Misclassification (GMA): Trigger causes misclassification to a target class.
Object Disappearance (BLINDING): Trigger causes objects to vanish.
Object Insertion (SPONGE): Trigger causes spurious detections.

Performance Metrics:

Total Models Evaluated: 288 models (18 per configuration).
Overall Accuracy: DistScan achieved an average detection accuracy of 96.99%.
Comparison: DistScan outperformed the best-performing applicable baseline by 27.32 percentage points on average.
- ODSCAN: Collapsed entirely in scene-level settings, achieving ~50% accuracy (random guessing) because its assumptions about trigger locality and specific class transitions were violated.
- MIA: Showed limited and inconsistent effectiveness, particularly failing on one-stage models and scene-level attacks that corrupted both regression and classification branches.
Robustness: DistScan maintained near-perfect detection (AUROC $\approx$ 1.0) across most configurations, with only minor performance degradation in complex COCO scenarios due to visually similar class pairs.

5. Significance and Claims

The paper claims that DistScan addresses a critical gap in object detection security by providing a principled, distribution-based defense primitive. Its significance lies in:

Generalization: It successfully detects backdoors without needing to know the trigger or the specific attack mechanism, making it effective against sophisticated scene-level attacks that defeat existing methods.
Simplicity and Efficiency: By leveraging the inherent statistical properties of the model's inference process (pre-NMS distribution), it avoids complex optimization or architectural modifications.
Future Direction: The authors hope this work draws attention to the underexplored threat of scene-level backdoor attacks and encourages further research into distribution-based detection signals as a broadly applicable defense strategy.

The authors explicitly state that their method relies on the alignment between a model's learned class prior and its training data statistics, a property that backdoor injection disrupts. This shift is observable on clean inputs alone, making DistScan a practical tool for inspecting models received from untrusted third parties.