💻 computer science

Class Activation Map Stability as an Uncertain Guide to Denoising Gain for Frozen Pedestrian Detectors Under Gaussian Noise

This study demonstrates that while the Attention-Preservation Score (APS) correlates with detection gains for specific reference detectors, it fails to serve as a reliable, generalizable guide for selecting denoising methods across diverse frozen pedestrian detectors under Gaussian noise.

Original authors: Vo Thanh Kiet, Rene Jaros, Ly Duc Minh, Petr Bilik, Radek Martinek

Published 2026-11-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Vo Thanh Kiet, Rene Jaros, Ly Duc Minh, Petr Bilik, Radek Martinek

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of computer vision, machines are taught to see the world much like humans do, but with a crucial difference: they rely on mathematical patterns rather than intuition. When a computer is trained to spot pedestrians in a crowd, it learns to recognize specific shapes, edges, and textures that signal a person is present. However, the real world is rarely perfect. Cameras capture images that are often grainy, blurry, or obscured by static, much like a photograph taken in a heavy fog. To help the computer see better, engineers often try to clean up these noisy images first, using software to smooth out the grain and sharpen the details. The hope is that a cleaner picture will lead to a smarter decision. But there is a catch. A picture can look beautiful and clear to a human eye while still confusing the computer, or conversely, a picture that looks messy might actually contain the exact clues the machine needs to make a correct call. The central question for researchers is whether cleaning an image actually helps the computer find what it is looking for, or if the cleaning process accidentally removes the very signals the machine relies on.

A team of researchers set out to solve this puzzle by testing whether a specific type of digital cleaning helps frozen pedestrian detectors—systems that have already been trained and cannot be re-taught. They focused on a method called an attention map, which is essentially a heat map generated by the computer to show where it is looking when it tries to find a person. If the computer is looking at the right place on a clean image, and then looks at the same place on a cleaned-up noisy image, the two maps should look very similar. The researchers called this similarity an "attention-preservation score." They wanted to know if a high score on this test meant the computer would actually find more people. To find the answer, they took 160 images containing people and mannequins, added different levels of digital noise to them, and then ran them through six different cleaning pipelines. These pipelines ranged from classic, rule-based filters to modern, learning-based systems that had been trained to remove noise. They tested these cleaned images against eight different computer detectors, using five different levels of noise intensity, from a light dusting of static to a heavy, blinding snowstorm of digital interference.

The results revealed a complex relationship between how an image looks and how a machine interprets it. The researchers found that for a specific, standard detector, the attention-preservation score did seem to track with success. When the computer's focus remained stable after cleaning, it was more likely to find the pedestrians. However, this connection was not a universal rule. When they pooled the results from all eight different detectors, the link between a stable attention map and a successful detection vanished. The score that worked for one machine did not predict the outcome for another. This suggests that the attention map is a measure of stability specific to each individual detector, rather than a reliable guide for choosing the best cleaning tool for any situation. The study also uncovered a hidden flaw in previous tests: some of the learning-based cleaning tools had left black, unprocessed strips at the edges of the images because they failed to cover the entire picture. When the researchers fixed this coverage issue and re-ran the tests, the performance of the learning-based tools improved, but the classic, rule-based filter known as BM3D still performed the best in seven out of the eight detectors at the highest noise levels.

Perhaps the most surprising finding came from a direct test of whether the computer was actually using the "important" parts of the image. The researchers took the areas where the computer was looking most intently and swapped those pixels with pixels from a different version of the same image. If the computer was truly relying on those specific high-attention areas to find the person, swapping them should have caused the detection to fail. Instead, the results were inconclusive. The swap did not produce a clear, measurable change in performance that could be attributed solely to the attention mechanism. This indicates that the computer's success is not driven by a single, isolated structural feature that can be easily isolated and tested. The study concludes that while attention maps are useful for understanding how a specific machine is behaving, they cannot be used as a standalone crystal ball to predict which image cleaning method will work best for a new detector. The choice of cleaning tool remains highly dependent on the specific machine doing the seeing, and the relationship between a clean image and a correct decision is far more fragile and variable than previously hoped.

Technical Summary: Class Activation Map Stability as an Uncertain Guide to Denoising Gain for Frozen Pedestrian Detectors Under Gaussian Noise

Problem Statement

Image denoising and pedestrian detection optimize distinct objectives. While denoising algorithms often improve image quality metrics like Peak Signal-to-Noise Ratio (PSNR) or Structural Similarity (SSIM), they do not necessarily improve the decision-making performance of a "frozen" (pre-trained and fixed) pedestrian detector. Existing approaches that train denoisers jointly with detection tasks couple these objectives, but real-world scenarios often require operating a separately trained denoising pipeline before a fixed detector.

The central hypothesis investigated is "structural preservation": the idea that denoising aids detection only if it preserves the specific image structures required by the downstream model. To test this, the authors examine whether the Attention-Preservation Score (APS)—a metric correlating the spatial activation maps of clean and processed images—can serve as a reliable guide for selecting denoisers that improve detection gain (measured by F1 score). The study addresses the uncertainty surrounding whether changes in Class Activation Maps (CAMs) reflect the removal of noise or the loss of useful content, and whether APS can predict detection improvements across different detector architectures and noise levels.

Methodology

The study evaluates eight frozen pedestrian detectors (five YOLO variants and three ResNet-based variants) across six processing pipelines (including Gaussian filtering, Block-Matching and 3D Filtering [BM3D], and three learned denoisers) and five levels of additive Gaussian noise ( $\sigma \in \{1, 5, 10, 20, 30\}$ ) on 160 images from the Person and Person-Like Objects (PnPLO) dataset.

Key Methodological Components:

Attention Map Definition: The study utilizes EigenCAM, which projects activations onto the principal component to generate a spatial activation summary without requiring a specific prediction target. The maps are oriented toward activation energy to handle polarity issues.
Attention-Preservation Score (APS): Defined as the average spatial Pearson correlation between the EigenCAM of a clean reference image and the processed (denoised) image. A companion statistic, APStgt, measures the mass of the attention map within ground-truth bounding boxes.
Denoising Gain ( $\Delta F1$ ): Calculated as the difference in F1 score between the denoised input and the raw noisy input for a fixed detector.
Experimental Controls:
- Geometry and Coverage: The authors identified and corrected "incomplete patch coverage" artifacts in learned denoisers (where border strips lacked reconstruction). They compared "archived" images (original outputs) with "repaired" images (regenerated with full coverage).
- Protocol Unification: A "unified protocol" was established where attention maps are hooked directly from the forward pass that produces detection boxes, replacing a previous "archived protocol" that used separate passes.
- Pixel Swap Probe: An exploratory intervention where pixels in high-attention regions of a denoised image (BM3D) were swapped into a different denoised image (Autoencoder) to test if high-attention areas are the primary drivers of detection gains.
Statistical Analysis: Nonparametric bootstrapping (5,000 replicates) was used to generate 95% confidence intervals for APS values and correlations. Partial correlations were computed to assess the association between APS and $\Delta F1$ after adjusting for noise levels and detector groups.

Key Contributions

Operational Definition of APS: The paper specifies APS as a detector-specific measure of spatial activation stability with explicit sign conventions and controls for preprocessing geometry, map polarity, and attribution choices.
Unified Evaluation Protocol: The study evaluates eight fixed checkpoints under a unified protocol, revealing that while APS and denoising gain show positive point estimates within specific detector groups, the pooled association across all detectors is near zero and highly dependent on the measurement protocol.
Coverage Artifact Correction: The authors regenerated three learned denoising pipelines to fix incomplete patch coverage. This repair increased the APS of learned denoisers by approximately 0.09 but did not significantly alter detection performance (F1), establishing that the coverage defect was an implementation artifact rather than a fundamental structural issue.
BM3D Performance: Under the unified protocol and full coverage, BM3D consistently maintained the highest APS at high noise levels ( $\sigma=30$ ) across seven of the eight detectors, outperforming learned denoisers in terms of activation preservation.
Exploratory Pixel Swap: The study conducted a controlled pixel swap experiment guided by attention. While the "high-attention" graft showed a small positive trend, the 95% confidence interval included zero, preventing a definitive claim that targeting high-attention pixels is the isolated mechanism for detection improvement.

Results

APS vs. Detection Gain: For the reference detector (YOLOv9m), APS and denoising gain were positively associated ( $r \approx 0.50$ to $0.61$). However, when pooling data across all eight detectors, the association dropped to near zero ( $r = +0.02$ , 95% CI: $-0.15$ to $+0.17$ ). This indicates that APS is detector-specific and not a universal predictor of denoising gain.
Impact of Coverage Repair: Fixing the incomplete patch coverage in learned denoisers raised their APS scores significantly (by +0.0896) but resulted in negligible changes to F1 scores (interval included zero).
Protocol Sensitivity: The "archived protocol" showed a negative pooled association between APS and gain at high noise, whereas the "unified protocol" showed a near-zero association. This highlights that the choice of measurement protocol significantly influences the observed relationship.
Pixel Swap: The primary pixel swap experiment (grafting high-attention pixels from BM3D into an Autoencoder output) yielded a micro-F1 increase of +0.0295, but the 95% bootstrap interval was $[-0.0195, +0.0824]$ , containing zero. Thus, the study could not isolate a structural mechanism where attention-targeted pixels alone drive detection gains.
BM3D Dominance: BM3D consistently achieved the highest APS at $\sigma=30$ for seven of eight checkpoints after coverage repair, suggesting it preserves the activation patterns of frozen detectors better than the tested learned denoisers under these specific conditions.

Significance and Claims

The paper modestly claims that APS is a detector-specific stability measure, not a validated universal denoiser selector.

Limitations of APS: While APS correlates with detection gain for a specific detector, this relationship does not generalize across different detector architectures or measurement protocols. The association is sensitive to grouping, score definitions, and preprocessing.
No Universal Mechanism: The study does not establish that preserving high-frequency structure or specific attention maps is the causal mechanism for improved detection. Factors such as coverage artifacts, polarity anchoring, and feature representation differences influence the scores.
Practical Implication: For the specific case of frozen pedestrian detectors under Gaussian noise, BM3D demonstrated superior activation preservation compared to the tested learned denoisers, even after correcting for implementation artifacts in the latter.
Future Requirements: The authors emphasize that broader utility claims or mechanistic conclusions require independent data, faithfulness checks for individual predictions, and testing against held-out conditions. The current results serve as an exploratory guide rather than a definitive rule for pipeline selection.