💬 NLP

UNIBROWSE: A Data-to-Agent Framework for Multimodal BrowseComp

The paper introduces UNIBROWSE, a novel data-to-agent framework that generates comprehensive training data covering all three multimodal browsing information-flow patterns and employs exploration-aware reinforcement learning to train a 35B-scale agent that achieves state-of-the-art performance on multimodal BrowseComp benchmarks, significantly outperforming both its base model and leading closed-source agents.

Original authors: Xiyu Wei, Qingwei Zong, Zhuocheng Yu, Sujian Li

Published 2026-07-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xiyu Wei, Qingwei Zong, Zhuocheng Yu, Sujian Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're a detective trying to solve a mystery, but instead of just reading a few clues on a notepad, you have to jump between a library, a photo album, and a live news feed to find the answer. That's what a "Multimodal BrowseComp" task is: a super-hard puzzle where an AI agent has to read text, look at pictures, and use search tools to piece together a solution that isn't just sitting on one webpage.

For a long time, the AI detectives we've been training were a bit one-sided. They were great at reading text clues or looking at a picture and then reading text to find an answer (like seeing a photo of a dog and searching "what breed is this?"). But they were terrible at the reverse: reading text to figure out what to look for in a picture (like reading a description of a weird building and then searching for a photo to confirm its shape).

The Big Discovery: The "UniBrowse" Detective
The researchers behind this paper, a team from Peking University, built a new training system called UNIBROWSE. Think of it as a super-smart training camp that finally teaches AI agents to handle all three types of clues:

Text-only: Just reading and connecting facts.
Image-to-Text: Starting with a photo and finding the story.
Text-to-Image: Reading a story first, then hunting for the matching photo to prove it.

Before this, most training camps only covered the first two. The authors argue that ignoring the third type (Text-to-Image) leaves the AI "undertrained" for real-world scenarios where you need to verify a fact visually after reading about it. They also noticed that previous methods were like trying to learn by wandering randomly through a city (which leads to lots of noise and wrong turns) or by only looking at a perfect, fake map (which doesn't match the messy real world). UNIBROWSE fixes this by starting with a solid "map" (a knowledge graph) but then sending the AI out to grab live, real-time evidence from the actual internet to make sure the clues are real.

The "Exploration" Filter: Cutting the Fluff
Here's the clever part: The researchers realized that not all practice problems are created equal. Some problems have only one boring way to solve them, while others force the agent to try different paths, back up, and try again.

To fix this, they invented a new metric called "exploration degree." Imagine you're grading a student's homework. If every student solves the math problem the exact same, easy way, you don't learn much. But if some students try a shortcut, others get stuck and backtrack, and a few find a clever new path, that's where the real learning happens.

The team used this "exploration degree" to filter their data. They threw away the boring, one-path problems and kept only the top 30% of the most "exploratory" questions. This made their Reinforcement Learning (the part where the AI learns by trial and error) much more efficient, focusing only on the puzzles that actually teach the agent how to think deeply.

The Results: A New Champion
Using this pipeline, they built a 35-billion-parameter AI agent (a very large brain). They didn't just guess it would work; they measured it against five different tough benchmarks.

The results were impressive. Their new agent, UniBrowse, scored an average accuracy of 54.4 across these tests.

This is a 10.5-point jump over its base model (Qwen3.5-35B-A3B), which started at 43.9.
It beat several famous "closed-source" giants (like GPT-5 and Gemini-2.5 Pro) which scored around 42.9 and 44.8 respectively.
It even outperformed other open-source models like Kimi K2.5 (50.8).

What They Don't Claim
It's important to know what this paper doesn't say. The authors are careful to note that while UniBrowse is the best open-source agent they've seen, it still hasn't beaten the absolute strongest proprietary systems (like Gemini-3.1 Pro, which hit 63.2). They also admit their agent is limited to public web browsing with text and images; it can't handle interactive websites, videos, or maps yet. They suggest that bigger models and better tools might be needed to close that remaining gap, but they don't claim to have solved the whole problem of AI browsing.

The Bottom Line
By building a training system that covers all three types of clues (text, image-to-text, and text-to-image) and filtering out the boring practice problems, the UNIBROWSE team has created an AI agent that is significantly better at solving complex, multi-step mysteries on the internet. It's a solid step forward, proving that how you train an agent matters just as much as how big its brain is.

Technical Summary: UNIBROWSE

Problem Statement

Multimodal BrowseComp tasks require agents to solve questions where answers are not directly available but must be composed from multiple pieces of dynamic web evidence. Unlike standard fact-seeking tasks, these problems demand long-horizon reasoning, tool use, and the integration of text and visual modalities. Real-world browsing involves three distinct information-flow patterns:

Text-only: Answers are composed solely from textual evidence.
Image-to-text: An initial image clue must be resolved to initiate textual browsing.
Text-to-image: Textual browsing narrows a target, requiring visual retrieval for final verification.

Existing data construction methods for training such agents suffer from three critical limitations:

Incomplete Pattern Coverage: Prior datasets (e.g., WebWatcher, Vision-DeepResearch) predominantly cover the image-to-text pattern, neglecting the equally important text-to-image pattern where textual reasoning precedes visual verification.
Paradigm Trade-offs: Methods relying on curated knowledge graphs lack web diversity, while those using random web walks introduce massive noise and false evidence.
Inefficient Training: Prior works apply reinforcement learning (RL) directly to unfiltered synthetic data, wasting computational resources on low-signal trajectories that lack diverse reasoning paths.

Methodology

The authors propose UNIBROWSE, a unified data-to-agent framework comprising a novel data construction pipeline and a training strategy for a 35B-scale multimodal browsing agent.

1. Data Construction Pipeline

The pipeline generates high-quality training data by bridging structured knowledge with live web retrieval:

Textual KG with Web Expansion: The process begins with a textual knowledge graph ( $G_{KG}$ ) to define structured reasoning paths. For each seed entity, a connected subgraph is constructed. To ensure realism, the pipeline augments these graph nodes with attributes extracted from live web retrieval, retaining only those corroborated by multiple search results.
Unified Multimodal Pattern Expansion: The pipeline evaluates generated evidence subgraphs to support all three information-flow patterns:
- Image-to-text: Attaches retrievable image nodes to initial seed entities.
- Text-to-image: Appends image nodes to the final answer entity if the entity possesses a distinct, visually verifiable attribute not present in the text.
- Text-only: Generates questions based purely on the layered evidence structure.
  These patterns can be combined within a single question (e.g., starting with an image and ending with visual verification).
Exploration-Driven Selection: To improve RL efficiency, the pipeline introduces an Exploration Degree metric ( $E(q)$ ). This metric measures the variance in the length of successful browsing trajectories across multiple rollouts. High exploration degree indicates that correct answers can be reached via diverse paths (e.g., direct vs. backtracking), providing richer contrastive signals for policy learning. Low-signal instances are filtered out.

2. Agent Training

The framework trains a 35B-scale agent (based on Qwen3.5-35B-A3B) in two stages:

Cold-Start Supervised Fine-Tuning (SFT): Uses ~8,000 tool-use trajectories to establish a stable action grammar and pattern-aware prior.
Exploration-Aware Reinforcement Learning (RL): Optimizes the SFT model using ~10,000 QA pairs selected from the top 30% of instances based on the exploration degree metric. The agent interacts with SEARCH (text, image-to-text, text-to-image) and LINKSUMMARY tools using a ReAct-style sequential decision process. The reward function combines answer correctness and format compliance, optimized via Group Relative Policy Optimization (GRPO).

Key Contributions

Unified Data Pipeline: UNIBROWSE is the first framework to simultaneously generate training data covering text-only, image-to-text, and text-to-image patterns, while augmenting knowledge graphs with live web retrieval for improved fidelity.
Exploration-Aware Filtering: The introduction of the exploration degree metric allows for the selection of high-value RL instances, significantly improving training efficiency by focusing on problems with rich, diverse reasoning signals.
State-of-the-Art Agent: The resulting UNIBROWSE agent achieves superior performance on multimodal browsing benchmarks, demonstrating the efficacy of the data-to-agent approach.

Experimental Results

The UNIBROWSE agent was evaluated on five diverse benchmarks: BrowseComp-VL (BC-VL), MM-BrowseComp (MM-BC), MMSearch+, MMSearch, and FVQA.

Performance: The final model (SFT + RL) achieved an average accuracy of 54.4% across the five benchmarks.
Improvement: This represents a 10.5 percentage point improvement over the base model (Qwen3.5-35B-A3B, 43.9%).
Comparison: The agent outperforms several closed-source workflows, including GPT-5 (42.9%), Gemini-2.5 Pro (44.8%), and Gemini-2.5 Flash (41.3%). It also surpasses prior open-source baselines like WebWatcher-32B (31.7%) and Vision-DeepResearch-30B-A3B.
Ablation Studies:
- Pattern Coverage: Training on all three patterns yielded significantly better results than training on text-only or image-to-text alone, particularly on the challenging MM-BC benchmark.
- Selection Strategy: Filtering RL data to the top 30% by exploration degree yielded the best performance; overly strict (top 10%) or loose (top 100%) filtering resulted in lower scores.

Significance

The paper posits that controllable multimodal data construction and exploration-aware post-training are essential for building effective browsing agents. By addressing the gap in pattern coverage (specifically the text-to-image modality) and optimizing the RL training signal through exploration diversity, UNIBROWSE demonstrates that open-source models can achieve competitive, state-of-the-art performance in complex, multimodal web reasoning tasks. The framework narrows the performance gap between open-source agents and the strongest proprietary workflows by more than half, validating the "data-to-agent" paradigm as a scalable path forward.