💻 computer science

GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection?

This paper introduces GenVideoLens, a fine-grained benchmark comprising 500 annotated videos across 15 authenticity dimensions, to reveal that while Large Vision-Language Models (LVLMs) perform well on perceptual cues, they significantly struggle with optical, physical, and temporal reasoning in AI-generated video detection.

Original authors: Yueying Zou, Pei Pei Li, Zekun Li, Xinyu Guo, Xing Cui, Huaibo Huang, Ran He

Published 2026-03-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yueying Zou, Pei Pei Li, Zekun Li, Xinyu Guo, Xing Cui, Huaibo Huang, Ran He

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to spot a fake painting in a museum. For a long time, the only way to test if a painting was real was to ask a simple question: "Is this fake or is it real?" If the detective guessed right, they got a gold star. If they guessed wrong, they got a red X.

But here's the problem: We don't know why they got it right or wrong. Did they spot a weird brushstroke? Did they notice the shadow was wrong? Or did they just get lucky?

This paper introduces a new tool called GenVideoLens to fix that. Instead of just asking "Real or Fake?", it acts like a super-powered magnifying glass that breaks the video down into 15 different clues to see exactly where the AI is failing.

Here is a simple breakdown of what the paper found, using some everyday analogies:

1. The Problem: The "Blind" Detective

Current AI models (called LVLMs) are like detectives who are great at spotting obvious fakes but terrible at understanding the physics of the world.

The Old Way: We just gave them a video and asked, "Is this real?" They would say "Yes" or "No." We knew their overall score, but we didn't know which part of the video they were lying about.
The New Way (GenVideoLens): We gave them a checklist of 15 specific things to look for, like "Do the shadows match the light?" or "Does the water splash like real water?"

2. The 15 Clues (The Checklist)

The researchers created a checklist divided into two categories:

A. The "Still Photo" Clues (Frame-Level)
These are things you can spot just by looking at one single picture.

Texture: Does the skin look like skin, or like smooth plastic?
Edges: Are the outlines of objects jagged and weird?
Lighting: Do the shadows point in the right direction?
Text: Is the writing on a sign gibberish?

B. The "Movie" Clues (Video-Level)
These are things you can only spot by watching the video move over time.

Physics: If a ball hits a wall, does it bounce back, or does it pass through it like a ghost?
Time: Does the person's face stay the same, or does it morph into a different person every second?
Logic: Is a lion walking inside a house? (That's physically possible but logically weird).

3. The Big Surprise: The "Smart but Clueless" AI

When the researchers tested 11 different AI models using this new checklist, they found some funny and scary results:

The "Surface-Level" Expert: The AIs were actually pretty good at spotting "ugly" details, like blurry edges or weird colors. They are like art critics who can tell if a painting looks "off" from a distance.
The "Physics" Fail: The AIs were terrible at understanding how the world works.
- Analogy: Imagine an AI watching a video of a person walking on water. The AI might say, "The water looks real, and the person's shoes look real, so this is a real video!" It completely misses the fact that people can't walk on water. It lacks "common sense physics."
The "Time Travel" Fail: The AIs struggled to understand time. If you shuffled the order of the video frames (like shuffling a deck of cards), the AI didn't notice! It treated the video like a slideshow of unrelated photos rather than a continuous movie.

4. The "Small vs. Big" Model Twist

Usually, we think bigger AI models are smarter. But this paper found something weird:

Sometimes, smaller, open-source models were better at spotting physical errors than the massive, expensive "Big Tech" models.
Why? The big models are so focused on "high-level" meaning (like "this is a happy scene") that they ignore the small, physical glitches. The smaller models are like a hawk, staring intensely at the tiny details and missing the big picture, which ironically helps them spot the fakes better in some cases.

5. The "Collapse" Effect

When the researchers asked the AI to check all 15 clues at once in a single question, the AI got confused.

Analogy: It's like asking a student to solve a math problem, write an essay, and draw a picture all in one sentence. The student just gives up and writes the same answer for everything. The AI started giving the same "Yes/No" answer for every single clue, even if the clues were totally different.

The Bottom Line

GenVideoLens is a diagnostic tool that tells us: "AI is getting really good at looking like a human, but it still doesn't understand how the human world actually works."

It's like a forger who can paint a perfect face but forgets that eyes need to blink and shadows need to move with the sun. This paper helps us build better detectors by teaching AI to pay attention to the laws of physics and time, not just the colors and shapes.

1. Problem Statement

With the rapid advancement of generative models (e.g., Sora, Open-Sora), AI-generated videos (AIGV) have become increasingly realistic, posing significant risks regarding misinformation and digital forensics. While Large Vision-Language Models (LVLMs) show promise in detecting such content, existing evaluation protocols are insufficient.

Limitation of Current Methods: Most benchmarks treat AIGV detection as a binary classification task (Real vs. Fake) and rely on coarse-grained metrics like overall accuracy or F1-score.
The Gap: These aggregate metrics fail to reveal why a model succeeds or fails. They do not diagnose specific weaknesses in reasoning capabilities (e.g., optical physics, temporal causality) or identify which specific authenticity cues models rely on. Consequently, it remains unclear which aspects of video authenticity LVLMs can reliably detect and where they fundamentally fail.

2. Methodology: GenVideoLens Benchmark

To address these limitations, the authors introduce GenVideoLens, a fine-grained benchmark designed for dimension-wise evaluation of LVLMs.

A. Dataset Construction

Scale: The dataset comprises 500 videos: 400 highly deceptive AI-generated videos and 100 real videos.
Sources:
- Real: Sourced from LOKI (real subset) and OpenVid-1M, filtered for high visual quality and coherence.
- Synthetic: Sourced from GenVideo, GenVidBench, and LOKI (synthetic subset, including Sora/Open-Sora outputs).
Filtering: A rigorous multi-stage process removed low-quality, easily detectable, or abstract content to ensure the remaining videos are "highly deceptive" and challenging for both humans and models.
Annotation: Over 6,000 expert annotations were collected by 7 graduate students with AI backgrounds. Annotations were aggregated using a margin-based voting strategy with expert review for uncertain cases.

B. 15-Dimensional Evaluation Framework

The core innovation is the decomposition of video authenticity into 15 distinct dimensions, categorized into two levels:

Frame-level (Perceptual & Static):
- Texture Authenticity
- Edge and Contour Clarity
- Material and Physical Consistency
- Local Forgery Artifacts
- Depth of Field and Background Reasonableness
- Text and Symbol Readability
- Naturalness of Composition
- Chromatic Consistency
- Lighting and Shadow Consistency
Video-level (Temporal & Causal):
- Inter-frame Consistency
- Facial and Movement Continuity
- Temporal and Scene Logic
- Physical Plausibility of Interactions
- Correctness of Reflection/Refraction/Parallax
- Real-world Logic Consistency

C. Experimental Setup

Models Evaluated: 11 representative LVLMs, including 9 open-source models (Qwen3-VL, InternVL, LLaVA families, BusterX++) and 2 proprietary models (GPT-5, Gemini-2.5-pro).
Metrics: Primary metric is the F1-score for the forged class ( $F_{forged}^1$ ). A secondary metric, Collapse Rate ( $C$ ), measures how often models output identical binary decisions across all dimensions, indicating a failure to perform dimension-specific reasoning.
Experiments:
- Dimension-wise performance analysis.
- Temporal sensitivity analysis (perturbing frame order and density).
- Physical/causal reasoning evaluation (using frame difference and optical flow maps).
- Cross-dimension prompting analysis.

3. Key Results

A. Pronounced Dimensional Imbalance

LVLMs exhibit significant performance variance across dimensions:

Strengths: Models perform relatively well on perceptual cues such as texture, edge clarity, and material consistency. Proprietary models generally outperform open-source ones here.
Weaknesses: Models struggle significantly with:
- Optical Consistency: Lighting, shadows, and reflection/refraction correctness.
- Physical Interactions: Contact, occlusion, and force plausibility.
- Temporal-Causal Reasoning: Inter-frame consistency, movement continuity, and scene logic.

B. The "Small vs. Large" Model Paradox

In specific dimensions (Optical consistency and Real-world logic), smaller open-source models sometimes outperform larger proprietary models.

Hypothesis: Proprietary models may over-rely on high-level semantic interpretations, causing them to overlook subtle physical inconsistencies. Smaller models may rely more directly on salient visual cues that align better with human annotations.

C. Limited Temporal Utilization

Temporal Perturbation: When frame orders were randomized or repeated, model performance remained nearly identical.
Dense vs. Sparse Inputs: Increasing the number of input frames did not improve performance.
Conclusion: Current LVLMs largely behave as frame-based visual analyzers rather than models capable of reasoning over temporal dynamics. They fail to utilize temporal dependencies effectively.

D. Failure in Physical and Causal Reasoning

Even when provided with auxiliary motion cues (frame difference maps and optical flow), models failed to detect physically impossible events (e.g., object penetration, impossible reflections). Their accuracy remained near random guessing (~50%), indicating a lack of true causal reasoning grounded in physical laws.

E. Cross-Dimension Collapse

When prompted to evaluate multiple dimensions simultaneously, models exhibited a high collapse rate (>90%), outputting identical decisions across all dimensions. This suggests they cannot disentangle different reasoning cues and suffer from interference when asked to reason about multiple aspects at once.

4. Key Contributions

GenVideoLens Benchmark: A fine-grained dataset with 400 deceptive AI videos and 100 real videos, annotated with over 6,000 labels across 15 authenticity dimensions.
Dimension-Wise Evaluation Framework: A novel protocol that moves beyond binary accuracy to systematically diagnose specific capability gaps in LVLMs.
Diagnostic Insights: Comprehensive analysis revealing that current LVLMs are weak in optical reasoning, physical interaction, and temporal causality, despite strong performance on static perceptual cues.
Guidance for Future Research: The findings highlight the need for future detectors to improve temporal modeling, physical constraint integration, and multi-cue reasoning capabilities.

5. Significance

GenVideoLens provides a critical "diagnostic lens" for the AI safety and forensics community. By shifting the focus from "how accurate is the model?" to "where does the model fail?", the paper:

Exposes Fundamental Limitations: It proves that current state-of-the-art LVLMs do not truly understand video physics or causality, relying instead on static visual patterns.
Guides Model Development: It suggests that future improvements should focus on temporal reasoning modules, physical world modeling, and training strategies that prevent "collapse" in multi-dimensional reasoning.
Enhances Trust: By identifying specific failure modes, it helps developers and policymakers understand the current reliability limits of AI-generated video detectors.