Technical Summary: WEBLICA - Scalable and Reproducible Training Environments for Visual Web Agents

Problem Statement

The development of autonomous visual web agents faces a critical bottleneck: the inability to scale training data and environments while maintaining reproducibility. The web is inherently complex, open-ended, and constantly evolving. Existing approaches suffer from significant limitations:

• Offline Trajectories: Methods relying on supervised fine-tuning (SFT) with offline human or model-generated trajectories lack the interactive nature required to handle the stochasticity of the web.
• Simulated Environments: Current simulated environments (e.g., WebArena) cover only a handful of manually defined domains, limiting generalization to the diversity of the real web.
• Live Web Training: Training directly on live websites is unstable due to timeouts, bot detection, and the inability to perform controlled ablations or ensure reproducible training conditions as websites change over time.

The central challenge is how to scale interactive web environments to capture web diversity without sacrificing the reproducibility necessary for rigorous training and evaluation.

Methodology

The authors propose WEBLICA (Web Replica), a framework designed to construct reproducible and scalable web environments for training visual web agents. The framework operates on two complementary mechanisms to generate diverse offline training environments:

1. HTTP-Level Caching (Weblica-Cache)

This mechanism captures and replays real-world website interactions with high fidelity.

• Recording: Using Playwright, the system records browsing sessions, capturing all HTTP requests and responses.
• Volatile Parameter Handling: A key innovation is the automated identification and normalization of volatile parameters (e.g., timestamps, session tokens) that cause cache misses during replay. The system uses a rule-based normalization pipeline to filter these parameters from URLs, headers, and POST bodies.
• Replay: By stripping volatile parameters, the system enables deterministic replay of complex, dynamic web interactions under complete network isolation.
• Scale: Leveraging the InstaV3 dataset, the authors matched tasks to cached environments, retaining 15,600 verified cached environments and tasks.

2. LLM-Based Environment Synthesis (Weblica-Synth)

To address the limitations of caching (which is restricted to domains where stable recordings exist) and to generate specific interactive capabilities, the authors employ an automated synthesis pipeline.

• Capability Extraction: The system analyzes successful and failed trajectories from a Qwen3-VL-32B agent to extract 19,721 fine-grained web interaction capabilities, aggregated into 144 higher-level capability groups (e.g., form submission, authentication, dynamic filtering).
• Generation: Using coding agents (Claude Code), the system generates self-contained, static web environments (HTML, CSS, JavaScript) without external framework dependencies.
  • Parameters: Generation is conditioned on a target capability group, a website category (sampled from 1,160 domains like banking, travel, etc.), and a visual style (sampled from 961 options).
  • Assets: Visual assets (images, banners) are generated using Z-Image-Turbo.
  • State Management: Since the environments are static, stateful interactions (e.g., shopping carts) are managed via JavaScript's localStorage.
• Validation: Generated sites undergo iterative self-validation via Playwright screenshots to ensure functionality and the absence of CSS issues.
• Scale: The pipeline generated 2,500 synthetic sites covering 44,227 tasks for training (Weblica-train) and 250 sites for validation (Weblica-val).

Agent Formulation and Training

• Observation & Action: The agent operates purely on visual input (1280x720 screenshots + URL) and predicts coordinate-based actions (click, hover) without relying on DOM structures or "set-of-marks" overlays, which can hurt generalization.
• Policy: The policy follows a ReAct-style framework, where a vision-language model (VLM) produces a reasoning trace before selecting an action.
• Training Pipeline:
  1. SFT Warm-Start: The base model (Qwen3-VL-Instruct) is fine-tuned on 51.7K verified successful trajectories.
  2. RL Training: The model undergoes Reinforcement Learning using Dr. GRPO (Group Relative Policy Optimization).
  3. Reward Mechanism: An LLM-as-Judge (GPT-4o) evaluates task success based on the task description, action sequence, and resulting screenshots, enabling training on open-ended tasks that lack programmatic verification.

Key Results

The authors evaluated their best model, WEBLICA-8B, against open-weight baselines and API-based agents across multiple benchmarks.

• Performance on Live Benchmarks: On the Online-Mind2Web benchmark, WEBLICA-8B achieved 39.2% pass@1 with only 30 steps. This outperforms open-weight baselines of similar size (e.g., MolmoWeb-8B at 35.3% with ≥100 steps) and uses significantly fewer inference steps.
• Test-Time Scaling: The model scales favorably with additional compute. With 240 total steps (pass@8), WEBLICA-8B reached 68.9% accuracy, approaching the performance of API-based models like Gemini CUA (60.8% at 100 steps).
• Comparison to Baselines:
  • WEBLICA-8B (30 steps) outperformed the base Qwen3-VL-8B by 10.8 percentage points on average across live benchmarks.
  • It surpassed the best open-weight baseline (MolmoWeb-8B) using 40% fewer total steps (60 steps vs. ≥100).
• Grounding Preservation: Despite training on data without explicit grounding supervision, WEBLICA-8B maintained or slightly improved performance on UI grounding benchmarks (MMBench-GUI, ScreenSpot-v2, ScreenSpot-Pro) compared to the base model, indicating that the gains stem from improved navigation behavior rather than degraded visual grounding.
• Ablation Studies:
  • Environment Composition: Synthesized environments generally outperformed cache-only environments on complex benchmarks like Online-Mind2Web and WebTailBench.
  • Training Stages: The combination of SFT and RL provided the best results across all model sizes (2B, 4B, 8B), with RL providing significant gains over SFT alone.

Significance and Claims

The paper claims that WEBLICA provides a scalable and reproducible alternative to live web training, effectively bridging the gap between the diversity of the real web and the stability required for large-scale Reinforcement Learning.

• Scalability: By combining HTTP caching with LLM-based synthesis, the framework generates thousands of diverse environments, overcoming the domain limitations of previous simulated environments.
• Reproducibility: The framework eliminates network latency, timeouts, and bot detection issues, allowing for controlled, deterministic training and evaluation.
• Efficiency: The approach demonstrates that high-performing visual web agents can be trained without relying on large-scale human demonstrations or expensive API models, achieving competitive results with significantly fewer inference steps.
• Generalization: The ability to train on synthetic environments that capture core navigation skills while maintaining visual grounding suggests a viable path toward generalist computer-use agents.

The authors acknowledge limitations, noting that cached environments are static snapshots and synthetic environments may not fully capture the complexity of dynamic real-world applications (sim-to-real gap). They also note the current focus on single-turn tasks, whereas real-world usage often involves multi-turn sessions and human-in-the-loop interactions.