Technical Summary: Region4Web and PageDigest

Problem Statement

Web agents perceive their environment through an observation space, yet the granularity of this observation has remained an underexamined design choice. Existing approaches typically treat the observation space at the same element-level granularity as the action space, representing the page as a flat or hierarchical list of individual DOM elements (e.g., AXTree nodes). While element-level granularity is natural for selecting specific actions (e.g., clicking a specific link), it is suboptimal for page state understanding.

In element-level observations, the functional organization of a page—how groups of elements serve shared purposes like "product listings," "navigation menus," or "search forms"—remains implicit. The agent is forced to infer these functional regions and their purposes from raw element signals at every step. This approach fails to leverage the structural cues that define functional units and often results in excessively long observation contexts that obscure the page's organization. Furthermore, reconstructing the full element-level observation at every step within a page is redundant, as most actions induce only marginal changes to the DOM.

Methodology

The paper proposes a two-part framework: Region4Web for constructing region-level observations and PageDigest for delivering these observations efficiently.

1. Region4Web: Constructing Region-Level Observation

Region4Web reorganizes the page's AXTree into functional regions through a two-stage process:

• Hierarchical Decomposition: This stage classifies every parent-child edge in the AXTree as either a merge or a cut in a single bottom-up traversal.
  • Input: A feature vector for each node $v$ ($x_v$) combining a learned role embedding with numeric structural features (depth, subtree size, child diversity, etc.).
  • Mechanism: An EdgeClassifier determines whether a child node should be separated from its parent. It uses the parent's features, the child's representation, and the mean representation of the parent's siblings as context. If the classification score exceeds a threshold $\tau$, the edge is cut, forming a new region boundary. Otherwise, the child is merged into the parent's region.
  • Output: A partition of the AXTree into subtrees, where each subtree constitutes a functional region.
• Semantic Abstraction: This stage interprets each identified region to make its purpose explicit.
  • Mechanism: A small, fine-tuned language model (Qwen3-0.6B) processes the preprocessed AXTree subtree of each region.
  • Output: Two orthogonal dimensions for each region:
    1. Purpose ($p_i$): A high-level description of what the region is for (e.g., "product listing," "search form").
    2. State Summary ($s_i$): A concise summary of the region's current actionable context (e.g., "promotional showcase for high-end sunglasses").

These stages are realized as small, dedicated models trained on annotations generated by a proprietary LLM (GPT-5-mini) from a dataset of 2,052 real-world web pages.

2. PageDigest: Web-Specific Inference Pipeline

PageDigest is a pipeline designed to deliver region-level observations to the actor agent as a compact digest that persists across steps within a page.

• Task-Relevant Region Selection: Upon entering a new page, the actor agent's backbone LLM receives the task instruction, action history, and the set of region abstractions $\{(p_i, s_i)\}$. The model selects task-relevant regions.
  • Selected Regions: Exposed as full AXTree subtrees to preserve element-level granularity for action execution.
  • Non-Selected Regions: Represented only by their purpose summaries, retaining the page's overall structural context without token bloat.
• Page-Aware Observation Transition Management: Instead of re-running Region4Web at every step, PageDigest tracks incremental changes.
  • It compares the current AXTree against the state at page entry to identify added, removed, or modified nodes.
  • Updates: Removed/modified nodes update the existing structure under their region purposes. Added nodes are listed separately to avoid shifting the purposes of existing regions.
  • This approach avoids redundant reconstruction while maintaining continuity of the page state.
• Fallback Mechanism: If the agent determines the selected regions are insufficient, it can invoke a view_all action to reveal all regions in their full AXTree form.

Key Contributions

1. Region4Web Framework: A novel method that reorganizes AXTrees into functional regions via hierarchical decomposition and semantic abstraction, making the page's functional organization explicit for the agent.
2. PageDigest Pipeline: A web-specific inference mechanism that delivers region-level observations as a compact, persistent digest, reducing observation length while tracking state transitions without full reconstruction.
3. Empirical Validation: Comprehensive evaluation on the WebArena benchmark demonstrating that region-level observation improves task success rates and reduces observation length across diverse backbone LLMs and agent methods.

Results

Evaluated on the WebArena benchmark (812 tasks across 5 domains: Shopping, CMS, Reddit, GitLab, Map), the proposed methods yielded the following results:

• Observation Length Reduction: PageDigest substantially reduced the average observation token length by 43% (from 6,437 to 3,671 tokens) across four different backbone LLMs (GPT-5.1, Gemini 3.1 Flash-Lite, DeepSeek-V3.2, Qwen3.5-27B). The reduction ranged from 16% to 54% depending on the backbone.
• Task Success Rate (SR) Improvement: Despite the reduced context, PageDigest improved the overall task success rate by 2.3 percentage points on average.
  • For example, with GPT-5.1, SR increased from 45.2% to 47.5%.
  • With Qwen3.5-27B, SR increased from 38.2% to 39.9%.
• Generalizability: The improvements held across both proprietary and open-source models of varying capacities, as well as established agent methods (SteP and AgentOccam), indicating that region-level observation provides a complementary signal for page state understanding independent of model strength.
• Ablation Studies:
  • Region4Web alone improved SR (48.5% $\to$ 50.3%) with comparable observation length, confirming that functional regions aid understanding.
  • PageDigest alone (without Region4Web, using element-level selection) lowered SR (48.5% $\to$ 46.1%), suggesting that element-level processing hinders performance even with compression.
  • Combined: The full PageDigest pipeline achieved the highest SR (53.9%) with the lowest observation length (3,814 tokens).
• Failure Analysis: Analysis of 50 failed trajectories showed that 82.0% of failures were attributable to the backbone agent's action selection capabilities, while only 8.0% were due to regressions within the PageDigest pipeline (decomposition, abstraction, or selection errors). This indicates the pipeline operates reliably as designed.

Significance and Claims

The paper argues that observation granularity is a critical, yet underexamined, design choice for web agents. By shifting from element-level to functional region-level observation, the authors demonstrate that:

1. Explicit Organization: Making the page's functional organization explicit (via regions and purposes) provides a more informative basis for page state understanding than inferring it from raw elements.
2. Efficiency and Performance: Region-level observation allows for a more compact context that does not sacrifice, and in fact improves, task success rates.
3. Decoupling: This approach separates observation design from model capability and action policy, offering a path toward more efficient web agents regardless of the underlying LLM's capacity.

The authors conclude that operating at the granularity of functional regions delivers a more compact and informative basis for decision-making than element-level processing alone, validating the hypothesis that how an agent "sees" the web is as important as how it "thinks" about it.