🤖 AI

LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings

This paper introduces LongWebBench, a comprehensive benchmark designed to evaluate the structural fidelity and functional executability of long-horizon webpage generation by vision-language models, revealing that current state-of-the-art systems struggle with long-range coherence and interactive capabilities despite visual plausibility.

Original authors: Yi Zhao, Zhen Yang, Mengpan Chen, Mingde Xu, Shanghui Gong, Xijun Liu, Jibing Gong, Jie Tang

Published 2026-06-17

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Yi Zhao, Zhen Yang, Mengpan Chen, Mingde Xu, Shanghui Gong, Xijun Liu, Jibing Gong, Jie Tang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a talented architect to build a house based on a single, massive photograph of a mansion.

The Problem:
Until now, most tests for AI "architects" (called Vision-Language Models) only asked them to build tiny, single-room cottages. The tests checked if the front door looked right or if the paint color matched. But real websites are like massive mansions with dozens of rooms, multiple floors, and complex wiring. If you ask an AI to build a whole mansion from one photo, it might get the front door right but forget how to connect the stairs to the second floor, or build a kitchen that looks real but has no working sink.

The Solution: LongWebBench
The authors of this paper created a new, tougher test called LongWebBench. Instead of just checking if the house looks like the photo, they check two things:

The Blueprint (Structure): Does the whole building make sense? Are the rooms in the right order? Is the roof connected to the walls?
The Plumbing & Electricity (Function): If you turn on the faucet, does water come out? If you flip the light switch, does the light turn on?

How They Built the Test:
They gathered two huge collections of real-world "mansion photos" (long webpages):

490 "Look-Alike" Tests: They took photos of very long websites (some so long you'd have to scroll for 30 screens to see the bottom). They asked AI models to recreate the code for these pages. The test checked if the AI could keep the layout consistent from the top of the page to the very bottom.
507 "Do-It" Tests: They picked 129 websites and gave the AI specific jobs, like "Find a flight to Tokyo," "Add an item to the cart," or "Filter the search results." The AI had to write code that actually did these things in a real web browser, not just pretend to do them.

The Results: The "Reality Gap"
When they ran the best AI models through this new test, they found some surprising things:

The "Long Scroll" Problem: As the websites got longer, the AI's ability to keep the structure correct got worse. It's like an architect who can design a perfect first floor but forgets how the second floor connects to it.
The "Fake House" Problem: Many AIs built websites that looked beautiful and realistic (like a movie set), but when you tried to click a button or fill out a form, nothing happened. The "plumbing" was broken.
The Input Issue: Even when the researchers broke the long photo into smaller pieces to make it easier for the AI to see, the AI still struggled to put the pieces together into a working whole.

The Takeaway
The paper concludes that we can't just judge AI by how pretty its drawings look. To be truly useful, an AI needs to be able to build long, complex websites that actually work when you interact with them. LongWebBench is the new ruler they are using to measure this, showing us that while AI is getting good at drawing, it still has a long way to go before it can build a fully functional digital house.

Technical Summary: LongWebBench

Problem Statement
Recent advances in Vision-Language Models (VLMs) have demonstrated the ability to generate front-end code (HTML, CSS, JavaScript) from visual inputs like webpage screenshots. However, existing evaluation benchmarks primarily focus on short, single-screen, and largely static webpages, measuring success through local visual similarity. This creates a significant "reality gap" when applied to real-world webpages, which often span multiple screens, contain repeated components, require style consistency across distant sections, and support complex, multi-step user interactions. Current evaluations fail to assess whether models can preserve global structure over long contexts or generate executable logic for goal-oriented tasks.

Methodology
To address this gap, the authors introduce LongWebBench, a benchmark designed to evaluate long-horizon webpage generation from both structural and functional perspectives. The benchmark consists of two complementary tasks:

Webpage Visual Fidelity Replication (W-VFR):
- Goal: Evaluate whether a model can reproduce the global visual structure of a long webpage.
- Dataset: 490 real-world long webpages (defined as exceeding three viewport heights at 1920×1080 resolution) across seven structural categories (e.g., informational articles, product pages, government pages).
- Evaluation Protocol: A multi-dimensional VLM-based metric (using GPT-4o) scores generated renderings against reference screenshots on a 0–10 scale across five dimensions: page scale/scroll consistency, global layout, section hierarchy, local visual styling, and information density. Local styling is further augmented by DINO-based feature similarity.
- Input Settings: Supports both single-image (full screenshot) and multi-image (split into viewport-sized tiles) inputs to distinguish between resolution constraints and long-context reasoning.
Webpage Functional Fidelity Realization (W-FFR):
- Goal: Evaluate whether a model can generate an executable webpage that supports specific user goals.
- Dataset: 507 goal-oriented interaction tasks distributed across 129 webpages, covering 13 functional categories (e.g., e-commerce, travel, data visualization).
- Evaluation Protocol: A DOM-augmented agent-based pipeline executes the generated code in a browser environment. A controller coordinates an "actor" agent to perform prescribed actions and a "critic" agent (powered by Gemini-3-Pro) to verify if the resulting DOM and page state satisfy the task specification.
- Metrics: Step Success Rate (SSR), Task Success Rate (TSR), and Page Success Rate (PSR).

Key Contributions

Benchmark Creation: The introduction of LongWebBench, the first benchmark to jointly evaluate long-range structural coherence and executable multi-step interactions for webpage generation.
Dataset Construction: The curation of 490 long webpages for structural fidelity and 507 interaction tasks for functional verification, ensuring diversity in layout, content density, and interaction complexity.
Evaluation Protocols: The design of complementary protocols: a VLM-based metric for structural fidelity and an agent-based execution pipeline for functional verification, validated through human agreement analysis and cross-judge robustness checks.
Empirical Analysis: A comprehensive evaluation of state-of-the-art open-source and proprietary VLMs under both single-image and multi-image settings.

Experimental Results
Experiments conducted on models such as GPT-5.2, Gemini-3-Pro, Claude-Opus-4.5, and various open-source models (e.g., GLM-4.6V, Qwen3-VL) reveal the following:

Structural Fidelity: Performance generally degrades as webpage length increases. While multi-image inputs alleviate resolution constraints, they do not fully resolve challenges in global layout planning and cross-segment integration. Even the best-performing models achieve only moderate structural fidelity scores on very long pages.
Functional Fidelity: Executable interaction is significantly more challenging than visual reconstruction. Models with high Step Success Rates (SSR) often exhibit much lower Task Success Rates (TSR) and Page Success Rates (PSR), indicating error accumulation in multi-step workflows. For instance, top models achieved SSRs above 86% but TSRs around 58–59%, with PSRs remaining below 22%.
Length Sensitivity: Both structural fidelity and functional success rates decline as the number of viewports (for structure) or interaction steps (for function) increases, highlighting the difficulty of maintaining state and logic over long horizons.
Evaluation Reliability: Automatic evaluation protocols showed strong agreement with human judgments (Pearson $r = 0.83$ for W-VFR; consistency >0.89 for W-FFR) and high robustness across different judge models.

Significance and Claims
The paper claims that LongWebBench highlights a critical limitation in current VLM capabilities: the inability to generate long webpages that are both structurally coherent and functionally executable. The authors argue that evaluating webpage generation based solely on static visual similarity is insufficient. Instead, executable interaction must be treated as a core criterion. The benchmark serves as a diagnostic testbed to reveal that while models can produce visually plausible outputs, they frequently fail to support the complex, multi-step user goals required in real-world web applications. The work underscores the need for future models to improve long-range structural reasoning and interaction logic synthesis.

Limitations
The authors note that LongWebBench focuses on stabilized, rendered frontends and excludes authentication, personalization, live backend services, and unbounded infinite scrolling. Consequently, the benchmark evaluates frontend structure and observable behavior rather than the full complexity of deployed web applications involving external databases or server-side logic.

Technical Summary: LongWebBench

More like this