On the Disagreement in Perturbation-based xAI -- Benchmarking Perturbation Choices for Flood Detection from SAR Images

Technical Summary: Disagreement in Perturbation-based xAI for Flood Detection from SAR Images

Problem Statement
Perturbation-based Explainable AI (xAI) methods are widely used to interpret deep learning models by altering input regions and measuring changes in class probabilities to generate relevance heatmaps. However, these methods are highly sensitive to design parameters. Previous research indicates that variations in perturbation settings—such as region geometry or replacement schemes—can lead to "disagreement," where different parameter choices yield contradictory relevance maps for the same prediction. This variability undermines the reliability of explanations, particularly in critical applications like Earth observation. While existing studies have noted this sensitivity, there is a lack of systematic characterization regarding how specific perturbation choices shape attributions, especially within the context of Synthetic Aperture Radar (SAR) imagery where replacement values carry physical meaning (e.g., backscatter intensity).

Methodology
The authors conduct a comprehensive benchmarking study grounded in the use case of flood detection from SAR imagery using a U-Net model trained on the Sen1Floods11 dataset. The study systematically investigates two key parameters of the perturbation pipeline:

1. Patch Geometry: The size and shape of the perturbed regions. The authors compare square windows of varying sizes (8×8, 32×32, 128×128 pixels) with different strides (overlapping vs. non-overlapping) against superpixel-based segments (generated via SLIC) that align with image structures.
2. Perturbation Type: The replacement scheme used to alter the input. Five strategies are evaluated:
   • Channel-wise minimum: Replaces patches with the minimum pixel value of the channel.
   • Class mean: Replaces patches with the mean value of a user-defined class (background).
   • Stochastic constant replacement: Samples a constant value from a local neighborhood distribution.
   • Patch substitution: Replaces patches with random patches from other images in the dataset.
   • Pixel shuffling: Randomly shuffles pixels within the perturbed region, preserving intensity statistics but destroying spatial arrangement.

The relevance estimation follows a four-step process: (I) generating locally perturbed images and inferring class probabilities; (II) computing probability difference maps; (III) defining an attribution area (typically the predicted water region); and (IV) calculating patch-wise relevance scores by averaging differences over the attribution area.

To evaluate the results, the authors employ:

• Relevance Distribution Analysis: Using signed distance transforms relative to water edges to visualize where relevance accumulates (water vs. background) across different strategies via 2D histograms.
• Faithfulness Evaluation (Correctness): A deletion-based approach where input regions are progressively removed (replaced by class-conditional means to simulate absence) according to their relevance ranking. The impact on the Dice score is monitored to determine if high-relevance regions are truly decision-critical.

Key Results
The study reveals significant disagreement in relevance maps based on parameter choices:

• Impact of Patch Geometry:
  • Size: Larger patches yield higher magnitude relevance scores due to larger prediction shifts but produce coarse, fragmented maps that obscure fine details. Smaller patches offer higher resolution and better localization of specific structures but result in lower magnitude scores.
  • Shape: Superpixel-based segments, which follow image boundaries, reduce the "aggregation effect" seen in square patches (where a single patch covers mixed classes) and produce relevance maps that align more closely with underlying image structures.
• Impact of Perturbation Type:
  • Channel Minimum: Acts as an extreme outlier (low backscatter), often suppressing informative water cues and amplifying contrast with the background. This leads to relevance maps dominated by background regions, regardless of the actual water content, resulting in low faithfulness (high AUC in deletion curves indicates the model does not rely on the top-ranked regions).
  • Class Mean, Stochastic Constant, and Patch Substitution: These strategies act as background surrogates. They show scene-dependent behavior; in water-heavy clusters, they can invert relevance signs (water becomes supportive vs. distracting). Their faithfulness varies with scene composition but generally tracks the model's reliance on texture and contrast cues.
  • Pixel Shuffling: Preserves pixel statistics but disrupts spatial coherence. It consistently highlights water-background boundaries rather than class interiors. While it produces high relevance for fragmented water bodies, this may reflect the model's sensitivity to boundary disruption rather than the water class itself, potentially leading to misleading interpretations of faithfulness.
• Cluster Analysis: The data points naturally cluster into five groups based on water coverage and body composition. The study finds that perturbation effects are not uniform; for instance, pixel shuffling severely degrades predictions for fragmented water (Clusters III-V) by destroying class separation, whereas it has a modest effect on homogeneous water bodies (Clusters I-II).

Significance and Claims
The paper argues that perturbation design is not merely a technical detail but a determining factor in relevance estimation. The authors demonstrate that:

1. Parameter Dependence: Different perturbation choices can steer relevance maps toward contradictory explanations (e.g., highlighting background vs. water, or boundaries vs. interiors).
2. Contextual Sensitivity: The "correct" perturbation choice depends on the specific dataset constraints and scene characteristics (e.g., SAR backscatter properties, water fragmentation).
3. Methodological Rigor: The study emphasizes that perturbation choices must be carefully inspected, evaluated, and justified as an integral part of the explanation process. Treating them as fixed or arbitrary parameters risks generating ambiguous or unfaithful explanations.
4. Evaluation Limitations: The authors note that standard faithfulness metrics (like deletion curves) can sometimes validate explanations for the wrong reasons (e.g., pixel shuffling appearing faithful because it disrupts boundaries the model relies on, even if the model's primary goal is water detection).

Ultimately, the work underscores the need for a robust understanding of how perturbation settings influence model behavior to ensure that xAI explanations in critical domains like flood detection are reliable and trustworthy.