Uncertainty-Aware Deepfake Detection via Multi-View Structural Learning

Technical Summary: Uncertainty-Aware Deepfake Detection via Multi-View Structural Learning

Problem Statement
Current deepfake detection methods, particularly those leveraging large-scale foundation models, often suffer from a critical limitation: they produce overconfident predictions when faced with distribution shifts, unseen manipulation techniques, or ambiguous inputs. While these models may achieve strong in-domain performance, their reliance on a single dominant representation (e.g., appearance artifacts or foundation-model embeddings) fails to capture the complexity of real-world forensic scenarios. Consequently, confidence estimates become unreliable, limiting the suitability of these detectors for operational deployment in security-critical biometric and forensic applications where accurate uncertainty quantification is as vital as detection accuracy.

Methodology: The DISCERN Framework
The authors propose DISCERN (Disagreement-Calibrated Evidential Reasoning Network), an uncertainty-aware framework that reframes deepfake detection as the identification of inconsistencies across complementary evidence sources rather than a binary classification based on a single feature space. The architecture integrates three distinct evidence streams:

1. Visual Stream: Utilizes a frozen CLIP ViT-L/14 backbone (with tunable LayerNorm parameters) to extract high-level facial representations. This stream leverages the strong cross-domain generalization capabilities of foundation models.
2. Semantic Stream: Models consistency among facial attributes, including expressions, action units (AUs), pose, and geometry. This stream employs differentiable constraints to evaluate predicates (e.g., alignment between a smile and AU12 activation). Discrepancies in these semantic relationships serve as indicators of manipulation.
3. Structural Stream: Captures class-dependent dependency patterns between semantic and forensic features using differentiable structural equation models (SEMs). It learns separate dependency graphs for real and fake samples; the detection signal is derived from the difference in reconstruction residuals between these class-conditioned models, effectively identifying shifts in structural dependencies rather than just feature values.

Uncertainty Modeling and Fusion
To integrate these heterogeneous signals, DISCERN employs a Dirichlet-based evidential framework. Each stream produces non-negative evidence vectors, which are fused to form a predictive distribution. A key innovation is the Inter-Branch Disagreement Calibration (IBDC) mechanism. IBDC explicitly measures the pairwise disagreement between the probability distributions of the three streams. It introduces a loss term that calibrates the model's epistemic uncertainty to match the observed level of inter-stream conflict. When streams agree, uncertainty is low; when they disagree, uncertainty increases, providing a reliable signal for selective prediction (abstaining from uncertain cases).

Key Contributions

• Multi-Stream Architecture: The proposal of a framework that jointly exploits visual, semantic, and structural evidence to improve robustness under distribution shift.
• Structural Dependency Modeling: A novel strategy that models class-conditioned relationships between semantic and forensic cues, providing information beyond conventional appearance-based representations.
• Inter-Branch Disagreement Calibration (IBDC): A mechanism that links predictive uncertainty directly to conflicts among heterogeneous evidence sources, moving beyond post-hoc calibration to integrate uncertainty modeling into the inference process.
• Comprehensive Evaluation: Extensive cross-dataset and real-world degradation evaluations demonstrating improvements in generalization, calibration, and selective prediction.

Experimental Results
Evaluated on the FaceForensics++ (FF++) training set and tested across five unseen benchmarks (Celeb-DF-v2, Celeb-DF-v3, DFDC, DFDC-Preview, and DFD), DISCERN demonstrates state-of-the-art performance:

• Calibration: The model achieves the lowest Expected Calibration Error (ECE) on four of five benchmarks and the lowest Excess Area Under the Risk-Coverage Curve (E-AURC) on all five. For instance, on Celeb-DF-v2, DISCERN achieves an ECE of 0.014, significantly outperforming the best baseline (0.029).
• Generalization: DISCERN attains the best frame-level AUC on three benchmarks and the best video-level AUC on three benchmarks. Notably, it shows substantial gains on Celeb-DF-v3 (which includes diffusion-based synthesis) and DFDC-Preview, suggesting the structural and semantic streams capture cues missed by single-stream adapters.
• Robustness: Under real-world degradations (noise, blur, JPEG compression, brightness, contrast), DISCERN achieves the highest mean AUC (90.93%), outperforming strong CLIP-based baselines.
• Ablation Studies: Removing the IBDC mechanism results in a significant drop in AUC (2.54 points) and a 3.9x increase in ECE, confirming that linking disagreement to uncertainty is critical for the framework's success.

Significance
The paper claims that combining complementary evidence with disagreement-aware uncertainty provides a robust foundation for trustworthy deepfake detection under distribution shift. By explicitly modeling inconsistencies across visual, semantic, and structural views, DISCERN offers a more reliable alternative to single-stream detectors, particularly in scenarios where inputs are degraded or manipulations are novel. The framework's ability to produce well-calibrated confidence estimates enables selective prediction, allowing systems to abstain from uncertain decisions—a crucial capability for high-stakes forensic and biometric applications. The authors note that this approach offers potential applicability to other high-stakes vision tasks requiring reliable uncertainty estimation.