Technical Summary: Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models

1. Problem Statement

The rapid advancement of AI-generated images (AIGI) has created a crisis in digital trust, necessitating robust detection mechanisms. While Multimodal Large Language Models (MLLMs) offer strong semantic reasoning and interpretability, they face a critical limitation in forensic tasks: the inability to detect low-level generator artifacts without compromising their pre-trained semantic capabilities.

Existing approaches often rely on external expert detectors to compensate for MLLMs' weaknesses or attempt to fine-tune MLLMs directly. However, the authors identify a fundamental representation conflict:

• Semantic vs. Artifact Learning: Pretrained MLLMs are optimized for high-level semantic features (style, content, logic). Forcing these models to learn subtle, low-level generator traces (artifacts) via standard fine-tuning (e.g., LoRA or full fine-tuning) disrupts the established semantic representations.
• The Dilemma: Standard fine-tuning fails to recover low-level artifacts effectively, while aggressive fine-tuning (full fine-tuning) leads to "catastrophic forgetting" of the model's core semantic knowledge, rendering it ineffective for general detection tasks.

2. Methodology: Deep Visual Residual MLLM (Deep-VRM)

To resolve the conflict between semantic preservation and artifact detection, the authors propose Deep-VRM, a novel architecture based on Deep Residual Injection.

2.1. Core Insight: Layer-wise Functional Stratification

Through linear probe analysis on pretrained MLLMs (specifically Qwen-2.5-VL-7B), the authors discovered a functional stratification within the model layers:

• Early-to-Middle Layers (Layers 1–16): These layers are critical for forming stable semantic representations. They converge rapidly on semantic tasks but remain stagnant when tasked with detecting low-level artifacts. Forcing these layers to learn artifacts disrupts semantic convergence.
• Later Layers: These layers are responsible for high-level reasoning and information integration.

2.2. Architecture Design

Deep-VRM employs a dual-branch visual architecture with a Residual Injection strategy to decouple the learning processes:

1. Frozen Semantic Path ($V_o$): The original visual encoder and the early layers of the LLM (up to layer $K-1$) remain frozen. This preserves the pre-trained semantic perception and prevents catastrophic forgetting.
2. Adapted Artifact Path ($V_a$): A parallel visual branch equipped with LoRA adapters is trained to extract artifact-specific visual cues.
3. Residual Injection: Instead of aligning features at the input level, the artifact features extracted by $V_a$ are injected directly into the intermediate hidden states of the LLM (at layer $K$).
   • The injection formula is: $\tilde{h}^{(K-1)}_v = \alpha \cdot h^{(K-1)}_v + \beta \cdot V_a(I)$, where $\alpha$ and $\beta$ are scaling factors (set to 0.5).
   • This "Green Road" bypasses the semantic-dominant early layers, allowing the model to learn low-level traces only after semantic features have converged.

2.3. Training Strategy

The training process is divided into two stages:

• Stage 1 (Semantic Alignment): Fine-tune the LLM component using a dataset ($D_1$) focused on semantic cues to activate pre-trained knowledge for AIGI detection.
• Stage 2 (Artifact-Aware Refinement): Introduce a dataset ($D_2$) focused on generator traces (created via VAE reconstruction to eliminate semantic interference). The early LLM layers remain frozen, while the residual branch ($V_a$) and the LLM layers from $K$ to $N$ are optimized. This allows the model to jointly model semantic reasoning and signal-level forensic cues.

3. Key Contributions

1. Discovery of Representation Conflict: The paper systematically reveals that native MLLMs cannot learn generalizable low-level artifacts without aggressively modifying the feature space, which compromises semantic extraction. It establishes that effective detection requires decoupling artifact learning from semantic preservation.
2. Deep-VRM Architecture: A novel method that injects artifact-specific features directly into deep layers of the LLM, bypassing the semantic-dominant early layers. This achieves optimal feature integration without external expert detectors.
3. State-of-the-Art Performance: The method achieves superior performance solely based on the MLLM, demonstrating robust generalization in "in-the-wild" settings where low-level artifacts are often degraded by post-processing.

4. Experimental Results

The authors evaluated Deep-VRM across multiple benchmarks, including GenImage, SynthBuster, AIGI-Bench, and WildRF (a dataset reflecting real-world social media conditions).

• Generalization: Deep-VRM achieved State-of-the-Art (SOTA) results across most benchmarks. For instance, on the GenImage dataset, it achieved an average accuracy of 97.42%, outperforming specialized forensic models like OMAT (94.63%) and DRCT (83.53%).
• Wild Robustness: On the WildRF benchmark, which tests robustness against compression and resizing, Deep-VRM achieved 89.51%, significantly outperforming the next best method (AIDE at 59.49%) and specialized forensic models.
• Comparison with MLLMs: Compared to general-purpose MLLMs (e.g., Qwen-2.5-VL-7B, LLaVA-1.5) which struggled with detection (50–68% accuracy) and specialized forensic MLLMs (Forensic-Chat, 88.75%), Deep-VRM achieved the highest average accuracy of 93.28%.
• Ablation Studies:
  • Injection Depth: Injecting at depth $K=16$ yielded the best balance. Shallow injection ($K=4$) disrupted semantic encoding, while very deep injection ($K=20$) left insufficient layers for joint reasoning.
  • Freezing Early Layers: Freezing early layers was crucial for preventing the forgetting of semantic knowledge, boosting performance on wild datasets by over 5% compared to full fine-tuning.
  • Fusion Coefficients: A balanced setting of $\alpha=0.5, \beta=0.5$ provided the best trade-off between preserving semantic representation and introducing artifact cues.

5. Significance and Claims

The paper claims that Deep-VRM successfully bridges the gap between high-level semantic analysis and low-level forensic perception. By resolving the representation conflict through Deep Residual Injection, the model enables MLLMs to autonomously detect both subtle generator traces and high-level semantic anomalies without relying on external expert detectors.

The authors emphasize that this approach provides a unified and generalizable framework for reliable AI-generated image detection. The model's ability to maintain robust performance under real-world image degradations (compression, resizing) suggests it does not rely on fragile, single-pattern artifacts but rather leverages a joint reasoning process. The work demonstrates that MLLMs can be adapted for forensic tasks effectively if their internal functional stratification is respected and leveraged through architectural innovation rather than brute-force fine-tuning.