🤖 AI

See More, Think Deeper: Query-Expanded Visual Evidence and Answer-Clue Guided Reflection for Long Video Understanding

The paper introduces CoVER, a framework that enhances long-video understanding in Video-LLMs by dynamically gathering query-expanded visual evidence to "see more" and employing answer-clue guided reflection to "think deeper," thereby outperforming existing models through evidence-centric and visually verifiable reasoning.

Original authors: Shuning Wang, Zhiheng Wu, YiNuo Lu, Naiming Liu, Chen Jia, Bowen Liu, Shuo Nie, Weijie Zhu, Yumeng Zhang

Published 2026-06-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shuning Wang, Zhiheng Wu, YiNuo Lu, Naiming Liu, Chen Jia, Bowen Liu, Shuo Nie, Weijie Zhu, Yumeng Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a mystery in a very long, complex movie. You are the detective, and the movie is your only source of clues.

Most current "AI detectives" (Video-LLMs) try to solve the mystery by watching the movie once, very quickly, and then guessing the answer. They might miss a tiny detail because they were looking at the whole screen at once, or they might guess based on what usually happens in movies rather than what actually happened in this specific scene.

The paper introduces a new detective named CoVER (Comprehensive Visual Evidence and Reflection). CoVER doesn't just guess; it follows a strict two-step process to "See More" and "Think Deeper."

Here is how CoVER works, using a simple analogy:

The Problem: The "Glance and Guess" Approach

Imagine you are asked, "What was the very first snack the character ate?"

Old AI: Watches the whole movie quickly. It sees a character eating an apple later on. It guesses, "It was an apple!" It didn't look closely enough to see that the character actually drank a soda with ice before eating the apple.
The Issue: The AI relies on a single, broad look and often misses the small, crucial clues hidden in the timeline.

The Solution: CoVER's Two-Step Detective Work

CoVER changes the game by using two special tools: The Magnifying Glass and The Fact-Checker.

Step 1: "See More" (The Magnifying Glass)

Instead of just watching the movie once, CoVER acts like a detective who realizes, "I need to look closer."

The Trick: Before answering, CoVER asks itself a bunch of follow-up questions, which the paper calls "pseudo-queries."
The Analogy: If the question is "What did they eat first?", CoVER doesn't just look for "food." It generates specific search terms like: "Is there a drink with ice?", "Are there watermelon slices?", "Is there a cup?"
The Result: It uses these specific questions to zoom in on tiny, high-definition clips from the video that it might have missed during the first quick glance. It gathers a complete pile of evidence before making a guess.

Step 2: "Think Deeper" (The Fact-Checker)

Once CoVER has gathered the evidence, it makes a Draft Answer (a first guess). But it doesn't stop there.

The Trick: CoVER takes its own draft answer and turns it into a "Clue" to test itself.
The Analogy: If CoVER guesses, "The first snack was watermelon," it creates a specific clue: "Check if the watermelon appears before the soda." It then goes back to the video specifically to look for that relationship.
The Result: If the video shows the soda came before the watermelon, CoVER catches its own mistake. It says, "Oh no, my draft answer was wrong because the evidence contradicts it," and it revises the answer to the correct one (the soda).

Why This Matters

Most AI models are like students who write an essay and hand it in immediately. CoVER is like a student who:

Researches the topic deeply (gathering more evidence).
Writes a first draft.
Critically reviews their own draft against the facts.
Fixes mistakes before handing in the final paper.

The Results

The paper shows that this method works. CoVER is much better at answering questions about long videos than other models of the same size. It even beats some very expensive, "closed-source" models (models you can't see inside) on certain tests.

In short: CoVER stops AI from guessing based on a quick glance. Instead, it forces the AI to hunt for specific clues, write a draft, and then double-check that draft against the video to ensure the answer is actually true.

Technical Summary: CoVER – Comprehensive Visual Evidence and Reflection for Long Video Understanding

1. Problem Statement

Recent Video Large Language Models (Video-LLMs) have advanced long-video understanding, yet they face two critical limitations in reasoning tasks:

Incomplete Evidence Acquisition: Existing methods often rely on a single search intent (the original question) to retrieve visual evidence. This coarse-grained approach frequently misses sparse, fine-grained, or critical visual cues (e.g., specific object attributes, temporal sequences, or state changes) necessary for complex reasoning.
Lack of Visual Verification: Current models typically generate answers in an open-loop manner. Once a draft answer is produced, the process terminates without verifying if the answer is supported by visual evidence or revising it if new evidence contradicts the initial hypothesis. This leads to a gap where high answer accuracy does not necessarily imply accurate evidence grounding.

2. Methodology: The CoVER Framework

The authors propose CoVER (Comprehensive Visual Evidence and Reflection), a framework designed to shift long-video understanding from answer-centric generation to evidence-centric, visually verifiable reasoning. CoVER operates through a closed-loop process involving two primary modules and a specialized retrieval tool.

2.1 Core Components

CueZoom (Cue-Aware Visual Evidence Zoom-In Tool):
A retrieval mechanism that locates high-resolution video segments based on textual cues. It computes relevance scores between video frames and textual cues (questions, pseudo-queries, or answer clues) using text and visual encoders. To ensure stability, it applies Laplacian Temporal Smoothing (LTS) to relevance scores, filtering out noisy peaks and merging neighboring frames into coherent evidence clips.
Query-Expanded Visual Evidence (QVE):
To address incomplete evidence acquisition, CoVER generates pseudo-queries ( $P$ ) from the original question and global video observation. These pseudo-queries decompose the reasoning demand into diverse, observable visual cues (e.g., "Is there a drink with ice cubes?"). The model uses the original question combined with these pseudo-queries to retrieve comprehensive, task-relevant zoom-in clips for generating a draft answer.
Answer-Clue Guided Reflection (AGR):
To address the lack of verification, CoVER introduces a reflection step. After generating a draft answer, the model derives an answer-specific visual clue (a verifiable condition that must hold if the answer is correct). This clue is used to retrieve specific verification evidence. The model then compares the draft answer against this new evidence. If the evidence contradicts the draft, the answer is revised; otherwise, it is preserved.

2.2 Training Pipeline

CoVER employs a two-stage Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL):

Stage 1 (QVE SFT): Trains the model to generate reliable draft answers using expanded evidence. The model learns an adaptive policy to invoke the zoom-in tool with pseudo-queries when global observation is insufficient.
Stage 2 (AGR SFT): Trains the model to verify draft answers using answer-specific clues. The loss function masks tokens corresponding to incorrect draft answers to prevent the model from learning from erroneous intermediate steps, focusing supervision on the verification reasoning and final answer generation.
Reinforcement Learning (GRPO): The model is further optimized using Group Relative Policy Optimization (GRPO) with a composite reward function comprising:
- Answer Reward: Encourages correct final predictions.
- Verification Reward: Penalizes revising a correct draft into an incorrect final answer and rewards consistent evidence-grounded transitions.
- Format Reward: Ensures adherence to the structured output format.

3. Key Contributions

The paper claims the following contributions:

First Visual-Feedback Reflection in Video-LLMs: CoVER is the first framework to integrate visual-feedback reflection into Video-LLM reasoning, transforming the inference process from open-loop to closed-loop.
Dual-Module Framework: It proposes a unified approach combining Query-Expanded Visual Evidence (to "See More" via diverse pseudo-queries) and Answer-Clue Guided Reflection (to "Think Deeper" via visual verification).
Evidence-Centric Reasoning: The framework shifts the paradigm from generating answers based on linguistic priors to reasoning based on comprehensive, verifiable visual evidence.

4. Experimental Results

Experiments were conducted on four major long-video benchmarks: MLVU, Video-MME, LongVideoBench, and LVBench. The base model used was Qwen2.5-VL-7B.

Performance Gains: CoVER-7B substantially outperformed the baseline Qwen2.5-VL-7B, achieving improvements of +3.9% on MLVU, +0.7% on Video-MME, +1.1% on Video-MME-Long, +2.3% on LongVideoBench, and +4.6% on LVBench.
Comparison with SOTA: CoVER surpassed state-of-the-art open-source reasoning models (e.g., Video-MTR, Conan) and even outperformed certain closed-source models on specific metrics, despite having a comparable parameter scale (7B).
Ablation Studies:
- Removing either Stage-1 or Stage-2 SFT resulted in performance degradation, confirming the necessity of both evidence acquisition and verification training.
- Removing Reinforcement Learning (RL) caused significant drops, particularly on MLVU and LVBench, highlighting the role of RL in optimizing evidence-grounded reasoning.
- Removing either the QVE or AGR modules consistently degraded performance across all benchmarks, demonstrating that both components are essential.

5. Significance and Claims

The paper posits that CoVER addresses the fundamental challenge of evidence grounding in long-video understanding. By moving away from reliance on salient partial cues or linguistic priors, CoVER enables models to:

Locate Sparse Evidence: Dynamically gather fine-grained visual facts through query expansion.
Verify Reasoning: Use visual feedback to validate and correct draft answers, ensuring the final output is supported by the video content.

The authors conclude that comprehensive evidence acquisition and visual-feedback reflection are critical for reliable long-video reasoning, offering a pathway to more robust Video-LLMs without necessarily increasing model scale.

6. Limitations

The authors acknowledge several limitations:

Dependency on Query Quality: The effectiveness relies on the quality of generated pseudo-queries and answer clues; ambiguous queries may retrieve misleading evidence.
Initial Answer Bias: Since reflection is conditioned on the initial draft, incorrect initial predictions may lead to biased verification clues.
Computational Overhead: The zoom-in retrieval process introduces extra computation compared to uniform sampling.
Holistic Understanding: The framework may be less effective for questions requiring holistic video understanding rather than localized visual evidence.