Non-identifiability of Explanations from Model Behavior in Deep Networks of Image Authenticity Judgments

1. Problem Statement

Deep Neural Networks (DNNs) have demonstrated the ability to predict human judgments on image authenticity with high accuracy. However, a critical gap exists between predictive performance and explanatory validity.

• The Core Issue: A model may successfully predict human ratings without using the same cues humans use, or it may rely on correlated proxies (e.g., image quality) rather than the target concept (authenticity).
• The Rashomon Effect: Even when multiple models achieve similar predictive accuracy, they often produce divergent explanations (attribution heatmaps) for the same input.
• The Question: Are the explanations generated by these models (e.g., via Grad-CAM, LIME) robust and consistent enough to serve as valid evidence for understanding the psychological mechanisms underlying human image authenticity judgments?

2. Methodology

Dataset and Preprocessing

• Dataset: The study utilized AIGCIQA2023, containing 2,400 AI-generated images rated by humans for Authenticity, Quality, and Correspondence.
• Filtering: Images from non-realistic categories (e.g., illustrations, arts) were excluded based on prompt metadata, resulting in a final set of 1,367 images.
• Reliability: Human ratings showed a split-half reliability of ~0.67, establishing a "noise ceiling" (maximum achievable correlation) of approximately 0.82.

Experimental Design

The authors employed a transfer-learning approach using frozen pre-trained backbones (to prevent overfitting on the small dataset) with a lightweight regression head trained to predict human Mean Opinion Scores (MOS).

• Architectures Tested: VGG-16, VGG-19, ResNet-152, DenseNet-161, EfficientNetB3, and a self-supervised Barlow Twins model.
• Variants: For each architecture, 10 independent models were trained using different random seeds and data splits to assess stability.
• Experiments:
  1. Exp 1: Standard train/val/test split to evaluate prediction accuracy and the relationship between authenticity and quality.
  2. Exp 2: Used the test set as an "explanation-optimization" set to refine channel pruning for attribution maps.
  3. Exp 3: Ensemble learning (Bagging and Stacking) to aggregate predictions and explanations.

Explanation and Consistency Metrics

The study evaluated explanation consistency using three methods:

1. Grad-CAM: Gradient-based attribution.
2. Multi-scale Pixel Masking (MPM): Perturbation-based occlusion.
3. LIME: Surrogate modeling using superpixel segmentation.

Consistency was measured in two dimensions:

• Within-Architecture: Correlation of heatmaps across the 10 random seeds for the same image.
• Across-Architecture: Correlation of "prototype" heatmaps (averaged across seeds) between different architectures.

Variance Decomposition: To determine if models tracked authenticity or just quality, the authors computed partial correlations: $r(A, \hat{A} | Q)$, measuring the residual correlation between true authenticity ($A$) and predicted authenticity ($\hat{A}$) after removing the linear contribution of quality ($Q$).

3. Key Results

A. Predictive Performance vs. Information Tracked

• Performance: Several models achieved strong predictive performance, reaching ~80% of the human noise ceiling (PLCC $\approx$ 0.62–0.63 for Barlow Twins and DenseNet).
• The Quality Proxy: A strong correlation ($r=0.84$) exists between human ratings of Quality and Authenticity.
• Divergent Learning:
  • VGG Models: Achieved moderate prediction scores but relied almost entirely on image quality cues. When quality variance was partialled out, their ability to predict authenticity-specific variance dropped to near zero ($r \approx 0.01$).
  • Barlow Twins / EfficientNetB3: Successfully tracked authenticity-specific variance (partial correlations $r \approx 0.15–0.24$), indicating they learned cues distinct from general image quality.

B. Explanation Consistency

• Within-Architecture Consistency:
  • Generally high for VGG, ResNet, and Barlow Twins (pixel-wise correlations $r > 0.98$).
  • Lower for DenseNet and EfficientNet.
  • Crucial Finding: Consistency was higher for images rated as more authentic. This suggests that highly authentic images contain robust, consistent cues that models reliably detect, whereas low-authenticity images may be explained by diverse, inconsistent features.
• Across-Architecture Consistency (The Core Failure):
  • Despite similar predictive performance, agreement between architectures was weak (average correlations often $< 0.35$).
  • VGG models produced systematically different heatmaps compared to other architectures.
  • Conclusion: The "Rashomon effect" is prevalent; different models solve the task using different visual evidence, making it impossible to identify a single "ground truth" explanation from a single model.

C. Ensemble Methods

• Performance: Ensemble models (Bagging and Stacking) outperformed all single models, achieving the highest alignment with human judgments (PLCC $\approx$ 0.73).
• Attribution: Ensembles allowed for the generation of image-level attribution maps (via MPM) that spatially segregated elements associated with high vs. low authenticity.
• Limitation: LIME could not effectively serve as a surrogate for the complex ensemble model, failing to capture the ensemble's attribution logic.

4. Key Contributions

1. Demonstration of Non-Identifiability: The paper provides empirical evidence that successful behavioral prediction in DNNs does not guarantee a unique or identifiable explanation. High-performing models can rely on different, non-overlapping visual cues.
2. Decoupling Quality and Authenticity: The study rigorously separates "quality" from "authenticity" in model learning, showing that many standard architectures (VGG) fail to learn authenticity-specific cues, instead exploiting the correlation with image quality.
3. Stability vs. Validity: It establishes that within-architecture stability (consistency across seeds) is a necessary but insufficient condition for psychological validity. A model can be internally consistent yet completely wrong about the cues humans use (e.g., VGG tracking quality instead of authenticity).
4. Ensemble Solution: Proposes that when cross-architecture consensus is lacking, ensemble models offer a path forward, improving predictive accuracy and providing a more robust basis for attribution, though they do not fully resolve the interpretability gap.

5. Significance and Implications

• For Cognitive Science: The results caution against using DNNs as direct proxies for human cognitive mechanisms. If a model predicts human behavior but highlights different image regions than other equally good models, its explanation cannot be assumed to reflect human psychology.
• For Explainable AI (XAI): The study highlights the limitations of current post-hoc explanation methods (Grad-CAM, LIME) in complex regression tasks. High predictive accuracy does not validate the explanation.
• For AI Safety/Forensics: In the context of detecting AI-generated images (Deepfakes), relying on a single model's "explanation" of why an image is fake is risky. The study suggests that robust detection requires ensemble approaches and a critical understanding that models may be "cheating" by using quality artifacts rather than semantic authenticity cues.

Final Conclusion:
The authors conclude that while deep networks can predict human behavior, post-hoc explanations from successful models should currently be considered weak evidence for cognitive mechanisms. The field must move beyond single-model analysis toward ensemble frameworks and rigorous validation of whether models track the specific variance of interest rather than correlated proxies.