Technical Summary: Visual-Seeker

Problem Statement

While Multimodal Large Language Models (MLLMs) have shown proficiency in various visual tasks, they frequently struggle with factual grounding in complex, open-world scenarios. Current multimodal deep search agents attempt to mitigate this by utilizing external tools, yet the visual-native search paradigm remains underexplored. Existing methods exhibit two primary limitations:

1. Passive Visual Perception: They often rely on simple images with explicit semantics or treat vision merely as a static input adjunct to text queries. This hinders the agent's ability to handle complex, real-world images containing multiple interconnected entities and fine-grained details.
2. Lack of Active Visual Evidence: Existing search trajectories are predominantly text-only or rely on simple image insertion. They lack active visual reasoning, where the agent dynamically harvests and integrates visual evidence from the web to support multi-hop, cross-modal reasoning.

Methodology: Visual-Seeker

To address these gaps, the authors propose Visual-Seeker, a visual-native multimodal deep search agent designed to perform active visual reasoning. The system operates by treating vision not as a static input but as a dynamic element where the agent actively attends to fine-grained details and harvests visual evidence throughout the search process.

1. Active Visual Reasoning Data Pipeline

A core component of the work is the synthesis of 5,000 high-quality multimodal search trajectories via a three-stage pipeline:

• Fine-grained Seed Entity Selection: Leveraging the LiveVQA dataset, the pipeline extracts structured entity information (name, description, type) from complex, multi-entity real-world images. It employs a three-step filtering strategy (Generic Mention Filtering, Complex Entity-Image Filtering, and Entity Disambiguation) to ensure seed entities are specific, visually grounded, and suitable for complex reasoning.
• Multi-hop Trajectory Synthesis: Using an offline Wikipedia knowledge graph, the system generates diverse, non-linear search trajectories. It employs a dual-strategy random walk:
  • Backtracking: Simulates revisiting previous assumptions to create tree-shaped reasoning branches.
  • Cycle Constraint: Generates disjoint branches that converge, ensuring strict cycle topologies to simulate complex browsing behaviors.
  • These trajectories are converted into text-based questions that conform to the subgraph constraints.
• Visual Evidence Injection (VEI): To force the model to engage in visual search, the pipeline locates answer entities and attributes, generates search keywords, and retrieves candidate images. An MLLM extracts fine-grained visual details (e.g., color patterns, spatial layouts) from these images. These details are merged into the original VQA as a "two-hop" QA pair, compelling the agent to perform text-to-image retrieval and pixel-level reasoning.

2. Agent Architecture and Training

• Workflow: Visual-Seeker operates in a ReAct-style loop, interacting with five external tools: text_search, reverse_image_search, search_image (for active visual evidence), visit (for webpage summarization), and image_crop (for fine-grained reasoning).
• Training Strategy: The model is trained using Supervised Fine-Tuning (SFT) on the synthesized 5K trajectories, avoiding costly reinforcement learning. A teacher model (Claude-4.6-Opus) generates the multimodal search trajectories. The training data includes:
  • 3,000 VQA instances without VEI.
  • 800 VQA instances with VEI.
  • 500 text-only QA trajectories.
  • 700 trajectories sampled from the FVQA training set.
• Base Model: The system is built upon Qwen3-VL-8B-Instruct.

Key Contributions

1. Visual-Seeker Agent: A novel multimodal deep search agent that unifies visual-native capabilities with search. It is capable of fine-grained visual perception in complex multi-entity scenes and proactively collects visual evidence for cross-modal search.
2. Active Visual Reasoning Pipeline: A data synthesis framework that enables models to develop fine-grained perception and active evidence collection. It generates 5K high-quality trajectories specifically designed to train agents in multi-hop visual reasoning.
3. State-of-the-Art Performance: The agent achieves superior performance across five challenging multimodal search benchmarks, outperforming existing open-source and proprietary models.

Experimental Results

The model was evaluated on five benchmarks: MMSearch, MMSearch-Plus, BrowseComp-VL, MM-BrowseComp, and VisBrowse-Bench.

• Overall Performance: Visual-Seeker achieved an average accuracy of 39.6%, surpassing all current multimodal deep search agents and competing with several proprietary models (e.g., GPT-5, Gemini-2.5-Pro).
• Comparison to Baselines: Compared to the base Qwen3-VL-8B-Instruct operating in an agentic workflow, Visual-Seeker demonstrated nearly double the performance across all benchmarks (an average improvement of +16.6%).
• Specific Benchmarks:
  • On MMSearch-Plus (complex multi-entity queries), the model showed significant improvement in visual understanding.
  • On MM-BrowseComp and VisBrowse-Bench (requiring visual evidence integration), Visual-Seeker outperformed proprietary models like GPT-5 and Gemini-2.5-Pro.
• Ablation Studies:
  • Data: The inclusion of trajectories with Visual Evidence Injection (VEI) was critical, yielding significant gains on benchmarks requiring visual evidence (e.g., +17.2% on MMSearch-Plus).
  • Tools: Removing either image_crop or search_image tools resulted in substantial performance drops, particularly on the VisBrowse benchmark, confirming the necessity of active visual grounding and evidence collection.

Significance and Claims

The paper claims that Visual-Seeker successfully bridges the gap between passive visual perception and active cross-modal search in open-world environments. By shifting the paradigm from treating vision as a static input to an active, reasoning-driven process, the agent demonstrates robust capabilities in:

• Identifying target entities within complex, entity-rich real-world images.
• Dynamically performing multi-hop reasoning.
• Proactively collecting and integrating visual evidence to answer queries that text-only or passive multimodal agents cannot solve.

The work validates that active visual reasoning is essential for the next generation of multimodal search agents, enabling them to navigate real-world web environments with a level of precision and grounding previously unattainable.