Technical Summary: OmniVL-Guard Pro

Problem Statement

Existing vision-language forgery detection and grounding methods predominantly operate under a closed-world paradigm. These self-contained Multimodal Large Language Models (MLLMs) rely solely on parametric memory and static training corpora. This approach faces a practical ceiling in dynamic, open-world forensic scenarios due to three primary constraints:

1. Finite Parametric Knowledge: Models cannot verify real-time events that occurred after their training cutoff or require external factual verification.
2. Static Perception: They lack the ability to perform fine-grained scrutiny of local boundaries, texture discontinuities, or subtle artifacts required for pixel-level forgery segmentation.
3. Limited Resolution: Native perceptual resolution is insufficient for detecting minute manipulations without external zooming or cropping tools.

Consequently, current methods struggle with Real-time Event Verification (requiring external clues) and Forgery Segmentation (requiring fine-grained local inspection). Furthermore, existing approaches often suffer from a "pseudo-success" pattern where the final answer is correct, but the reasoning process is distorted or lacks a verifiable evidence chain.

Methodology

The authors propose OmniVL-Guard Pro, a tool-augmented agent designed to shift unified forensics from closed-world prediction to open-world, evidence-driven reasoning. The framework integrates a specialized tool environment with a novel training pipeline.

1. Tool-Enhanced Environment

The agent operates within an environment equipped with seven specific tools to acquire external knowledge and fine-grained visual data:

• Real-time Event Search: Queries the web for time-sensitive facts.
• Local Cropping & Zooming: Extracts and enlarges specific regions for detailed inspection.
• Edge-Anomaly Screening: Scans for local edge density, Laplacian energy, and neighborhood discrepancies to identify suspicious regions.
• Face Detection: Identifies facial bounding boxes for portrait inspection.
• Video Frame Extraction: Extracts frames from specific temporal segments for image-level analysis.
• SAM3 Segmentation: Generates pixel-level masks based on prompts (boxes, points).

2. Data Construction: Tree-Structured Self-Evolving Tool Trajectory Generation

To train the agent, the authors construct the Full-Spectrum Tool Reasoning (FSTR) dataset. They address the "efficiency–bias dilemma" (where free exploration yields low-quality data, and ground-truth injection causes hindsight bias) through a four-stage process:

• Seed Guidance: Uses an MLLM expert pool (Explorer and Guider) to perform tree-structured search. The Guider validates steps against the ground truth, pruning invalid branches.
• Guider-Free Self-Evolving Expansion: The policy autonomously explores the tool environment without answer-aware guidance, scaling up the dataset through iterative rollouts.
• Weakly-Hinted Hard Trajectory Synthesis: For long-tail hard samples that fail self-evolution, a Guider provides weak hints (e.g., rough regions or keywords) to guide the Explorer without revealing the answer. A Refiner then rewrites the reasoning to eliminate explicit hint traces, ensuring the final trajectory appears autonomous.
• Dataset Composition: The resulting FSTR dataset includes $FSTR_{sft}$ (69,323 samples) for cold-start Supervised Fine-Tuning (SFT) and $FSTR_{rl}$ (81,087 samples) for Reinforcement Learning (RL).

3. Training Strategy: Checker-Guided Agentic Reinforcement Learning (CGARL)

The training follows a three-stage process to ensure both answer correctness and reasoning consistency:

• Stage I (Outcome-Driven RL): The model is trained on $FSTR_{sft}$ via SFT, followed by outcome-level RL on a subset of $FSTR_{rl}$ using ARSPO. This yields a policy ($\pi_{ans}$) capable of invoking tools and producing correct answers.
• Stage II (Checker Construction): A Checker model is trained via distillation. It evaluates tool trajectories produced by $\pi_{ans}$ and assigns a discrete score ($s \in \{0, 0.5, 1\}$) based on internal consistency, tool usage reasonableness, and evidence sufficiency. This explicitly penalizes the "pseudo-success" pattern.
• Stage III (Process-Supervised RL): The main policy undergoes Checker-guided RL on the remaining $FSTR_{rl}$. The reward function combines answer correctness with the Checker's process score ($s_{proc}$) and a tool-use efficiency reward. This encourages the model to generate coherent, evidence-consistent reasoning trajectories.

Key Contributions

1. OmniVL-Guard Pro: An open-world, tool-augmented agent that unifies forgery detection, grounding, and segmentation across text, image, and video modalities, supporting real-time event verification and fine-grained segmentation.
2. FSTR Dataset: A comprehensive dataset comprising high-fidelity tool-search trajectories ($FSTR_{sft}$) and RL training data ($FSTR_{rl}$), constructed via a novel self-evolving framework that mitigates hindsight bias.
3. Checker-Guided Agentic RL (CGARL): A training paradigm that introduces process-level supervision to ensure the reasoning process is consistent with the final judgment, effectively addressing the "answer is correct but process is distorted" failure mode.
4. RealFact Benchmark: A new benchmark for real-time event verification covering events from January 2025 to March 2026, designed to test capabilities beyond parametric memory.

Experimental Results

The model was evaluated on both in-domain and out-of-domain (OOD) benchmarks, including ISOT, CASIA2.0, MMFakeBench, FakeTT, and IMD.

• In-Domain Performance: OmniVL-Guard Pro achieved state-of-the-art performance across all tasks. Notable improvements over the previous best (OmniVL-Guard) included:
  • RealFact Verification: +30.71% accuracy.
  • Segmentation (Dice): +25.77%.
  • Text-Image Localization: +30.71%.
• Zero-Shot Generalization: The model demonstrated strong OOD performance, outperforming general MLLMs (e.g., Qwen3VL-235B) and domain-specific baselines on unseen datasets. For instance, it achieved 81.74% accuracy on Text-Image tasks and 42.93% Dice on segmentation tasks without fine-tuning.
• Ablation Studies:
  • Removing the Checker or the RL stages resulted in significant performance drops, confirming the necessity of process supervision.
  • Removing specific tools (e.g., Search, Video Frame Extraction) caused substantial degradation in corresponding tasks (e.g., RealFact accuracy dropped from 86.5% to 62.3% without Search).
  • The Weakly-Hinted Hard Trajectory Synthesis was critical, with its removal causing performance drops of up to 9.63% on text localization.

Significance and Claims

The paper claims that OmniVL-Guard Pro represents a paradigm shift in vision-language forensics, moving from closed-world prediction to open-world evidence-driven reasoning. By integrating external tools and enforcing process-level consistency through CGARL, the model transcends the intrinsic limits of parametric knowledge and static perception. The authors assert that this approach not only achieves state-of-the-art performance but also establishes a robust framework for verifying real-time events and localizing fine-grained manipulations in a dynamic world. The work emphasizes that simply scaling model size is insufficient; instead, equipping models with the ability to actively seek, inspect, and reason over external evidence is the key to advancing forensic capabilities.