Technical Summary: OmniVL-Guard

Problem Statement

The rapid evolution of generative AI has lowered the barrier for creating convincing misinformation, posing severe risks to multimedia information security. Existing forgery detection methods are predominantly confined to uni-modal (e.g., solely images or text) or bi-modal (e.g., image-text) settings. This fragmentation fails to address the reality of modern social media, where misinformation often involves interleaved text, images, and videos simultaneously.

Furthermore, developing a unified framework for "omnibus" vision-language forgery detection and grounding (localizing manipulated regions) presents a critical "difficulty bias" problem. In multi-task optimization, the simpler veracity classification task tends to dominate the gradients, leading to suboptimal performance in fine-grained grounding tasks (e.g., localizing tampered text indices, image bounding boxes, or video time segments). Standard Supervised Fine-Tuning (SFT) fails to yield consistent gains across these diverse modalities due to insufficient reasoning capabilities, while existing Reinforcement Learning (RL) approaches often suffer from training dynamics dominated by easier sub-tasks.

Methodology

The authors propose OmniVL-Guard, a unified framework designed to handle text, images, and videos within a single paradigm. The framework integrates two core components:

1. Self-Evolving CoT Generation

To address the "cold-start" challenge of generating high-quality Chain-of-Thought (CoT) data for forensic reasoning, the authors introduce a four-stage pipeline:

• Source Data Collection: Aggregating public datasets covering text, image, video, and image-text modalities.
• Forensic Reasoning Seed Priming: Using state-of-the-art Multi-modal Large Language Models (MLLMs) to generate initial reasoning paths, which are rigorously filtered for consistency and correctness to create a seed dataset ($D^0_s$).
• Seed Bootstrapping: An iterative self-evolution loop where the model generates CoT for remaining data, which is validated and merged to expand the dataset ($D^k_s$). This process runs for three iterations to avoid local optima.
• Collaborative Hard-CoT Synthesis: A multi-agent workflow to handle long-tail hard samples. One agent generates CoT conditioned on Ground Truth (GT), a second "Refiner" rewrites it to simulate natural deduction (removing hindsight bias), and a third validates the logic.
• Result: The construction of the Full-Spectrum Forensic Reasoning (FSFR) dataset, comprising high-quality CoT annotations for SFT cold-start and data for RL optimization.

2. Adaptive Reward Scaling Policy Optimization (ARSPO)

To resolve the difficulty imbalance in multi-task RL, the authors propose ARSPO. Theoretical analysis reveals that for high-difficulty tasks (like localization), the gradient sensitivity ($H'_k$) is often small, causing the model to plateau. ARSPO addresses this through:

• Task-Based Reward Mapping: Instead of linear mapping, the framework uses an exponential reward function ($g_k(x) = e^{ax}$) for fine-grained localization tasks. This convex mapping amplifies the gradient contribution of high-quality responses, compensating for the vanishing gradients caused by task difficulty.
• Dynamic Coefficient Adjustment: A mechanism that periodically monitors learning trends and adjusts task weights ($l_{k,s}$). It identifies "laggard" tasks (those with the slowest progress) and dynamically boosts their coefficients while decaying those of mastered tasks, ensuring balanced joint optimization.

Key Contributions

1. Unified Framework: The first work to propose a unified Vision-Language Model (VLM) framework capable of processing interleaved text, images, and videos for both forgery detection and fine-grained grounding, pioneering the integration of RL for forensic reasoning.
2. FSFR Dataset: A comprehensive multimodal corpus designed for the full SFT-RL-Test pipeline, constructed via a Self-Evolving CoT Generation pipeline to synthesize high-quality reasoning paths.
3. ARSPO Algorithm: A dynamic balancing algorithm that effectively resolves exploration imbalance in multi-task RL, significantly boosting performance on high-difficulty tasks without sacrificing performance on easier ones.

Experimental Results

Extensive experiments were conducted on In-Domain test sets and Out-Of-Domain (OOD) benchmarks (including ISOT, CASIA2.0, MMDFakeBench, and FakeSV).

• In-Domain Performance: OmniVL-Guard achieved state-of-the-art performance across all modalities. Notably, it outperformed the best existing methods by significant margins in fine-grained localization:
  • Video Temporal Localization: +37.79% improvement.
  • Text Localization: +22.92% improvement.
  • Image Localization: +5.73% improvement.
  • It also maintained highly competitive binary classification accuracy (96.20%).
• Zero-Shot Generalization: The model demonstrated robust zero-shot generalization on OOD scenarios, significantly surpassing domain-specific models that suffer from overfitting. For instance, it achieved 79.38% accuracy on Text-Image tasks and 63.55% on Text-Video tasks.
• Ablation Studies:
  • Removing the Collaborative Hard-CoT Synthesis led to consistent performance degradation, particularly in localization tasks (e.g., a 7.47% drop in Text Localization).
  • Direct GT injection (bypassing the self-evolution process) resulted in substantial performance gaps, confirming that avoiding "hindsight bias" in reasoning paths is critical.
  • The exponential reward mapping ($a=3$) was found to be optimal; linear mappings failed to amplify gradients for difficult tasks, while overly steep mappings induced noise.

Significance and Claims

The paper claims that OmniVL-Guard underscores the importance of addressing task imbalances in multi-modal forensics. By identifying the "difficulty bias" in joint optimization, the authors demonstrate that balanced reinforcement learning, driven by self-evolving reasoning and adaptive reward shaping, is essential for achieving robust forensic capabilities.

The work provides a scalable foundation for future research in content integrity, moving beyond fragmented uni-modal defenses toward a unified approach capable of handling the complex, hybrid nature of modern deepfakes. The authors emphasize that their framework effectively overcomes the generalization fragility of traditional models against unseen forgeries, offering a pathway to more reliable digital information security.

Note: The authors acknowledge the potential dual-use risk of their advanced reasoning capabilities and state that the dataset will be shared exclusively for academic purposes via a rigorous application-and-review process.