Incentivizing Vision Language Models to Search for Long Video Question Answering

The system utilizes a dual-indexing infrastructure:

• Semantic Visual Index: Videos are segmented into 8-second clips and embedded using the ViCLIP encoder.
• Subtitle Index: Text transcripts are mapped to exact frame windows, allowing the agent to search for specific spoken dialog or named entities that visual embeddings might miss.

2.2 VETL: Visual Evidence via Temporal Logic

To solve the reward sparsity problem, the authors propose VETL, a neuro-symbolic pipeline that bridges open-ended natural language with discrete visual verification.

• Specification Generation: Complex user queries are compiled into formal Temporal Logic (TL) specifications. An LLM generator proposes a TL formula $\phi$ over a set of atomic visual primitives (e.g., "woman," "pours water," "adds strawberry"). A judge model verifies this specification for entity coverage and temporal faithfulness.
• Decoupling Semantics and Time: The TL specification disentangles the visual grounding events (primitives) from their temporal dependencies. This transforms the long-horizon question into an exact checklist of required visual elements.
• Dense Reward Signal: During RL post-training, the agent's retrieved video context is evaluated against the TL specification. A deterministic visual checker (using ViCLIP) detects if specific primitives are present in the retrieved frames. The reward $R_{VETL}$ is calculated as the fraction of required primitives successfully observed across the entire trajectory. This provides dense, step-wise feedback, incentivizing the agent to gather specific evidence rather than just guessing the final answer.

2.3 Reinforcement Learning

The framework employs Group Relative Policy Optimization (GRPO) for post-training. The total reward is a weighted combination of the final answer accuracy (Exact Match, $R_{EM}$) and the VETL evidence reward ($R_{VETL}$):
$$R = \lambda_1 R_{EM} + \lambda_2 R_{VETL}$$
where $\lambda_1=0.7$ and $\lambda_2=0.3$. This objective encourages the model to learn a search policy that rigorously verifies context before answering.

3. Key Contributions

The paper outlines four primary contributions:

1. VSeek Framework: An agentic system that converts LVQA into an active, multi-turn discovery process using natural language as the primary search interface, bypassing the need for rigid timestamp prediction.
2. VETL Pipeline: A neuro-symbolic method that converts open-ended questions into verifiable temporal-logic specifications, effectively creating "visual unit tests" for context construction.
3. Verified RL Rewards: A novel reward shaping mechanism that scores an agent's evidence-gathering trajectory against extracted visual primitives, enabling checkable, step-wise feedback beyond outcome-only supervision.
4. Empirical Validation: Demonstration that optimizing for verified context construction significantly boosts long-horizon agentic reasoning performance.

4. Experimental Results

The authors evaluated VSeek on multiple in-domain (LongVideoBench, MLVU, Video-MME) and out-of-domain (LVBench, CGBench-mini) benchmarks.

• Performance Gains: Compared to base models (Qwen3-VL-4B-Thinking), VSeek variants showed significant improvements. Specifically, VSeek-VETL improved Pass@1 scores by up to 8% and Pass@4 scores by 15% on long-video understanding benchmarks.
• Reward Effectiveness: The introduction of VETL rewards provided a distinct advantage over models trained with only Exact Match rewards (VSeek-EM), particularly on in-domain datasets where evidence coverage is critical.
• Long-Video Robustness: As video duration increased (e.g., >10 minutes), VSeek variants maintained higher reasoning stability compared to baselines like VideoTree and Naive RAG. VSeek-VETL specifically learned to produce highly specific search keywords, avoiding the generic queries that often lead to irrelevant clips.
• Efficiency: VSeek variants achieved state-of-the-art performance while processing significantly fewer frames on average (approx. 30–32 frames) compared to the fixed 64-frame budget of standard VLMs and RAG baselines.
• Algorithmic Robustness: The benefits of verified rewards were consistent across GRPO and RLOO algorithms, though degradation was observed with GSPO.

5. Significance and Claims

The paper claims that the path forward for agentic long-video QA must mirror breakthroughs in LLM reasoning for mathematics and programming, where progress is catalyzed by verifiable rewards.

• Shift from Passive to Active: VSeek moves the field away from passive, uniform sampling toward active, iterative evidence seeking, acknowledging that humans do not rely on random frame sampling for long documents.
• Bridging the Verification Gap: By introducing VETL, the authors claim to have solved the critical bottleneck of applying RL to video: the lack of a signal to verify context construction. This allows for training agents that are incentivized to find the "correct" visual evidence, not just the "correct" final answer.
• Foundation for Future Agents: The authors position VSeek and VETL as a foundational step for stronger video agents, suggesting that explicit optimization for verifiable evidence-seeking behavior is essential for scaling long-video understanding.

The paper concludes that while challenges remain (such as sequential search limitations and retriever bottlenecks), the combination of agentic search and neuro-symbolic verification offers a robust path toward solving complex, long-horizon video reasoning tasks.