评测长网页生成的结构与功能，发现视觉好看不等于能用。
LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings
- 构建长网页生成的双维度评估框架，兼顾结构与功能。
- 实测发现网页越长，结构保真度越差，且多步操作常失败。
- 适合关注长网页生成真实可用性的研究者和开发者。
近期视觉语言模型（VLMs）在从视觉输入生成网页方面取得进展，但现有评估主要聚焦于短、单屏、静态网页。本文提出 LongWebBench，一个面向长时序网页生成的基准测试，从结构与功能两个角度进行评估。该基准包含490个真实长网页用于结构保真度评估，以及129个网页上的507项目标导向交互任务用于功能评估。采用两种互补评估协议：基于多维VLM的指标评估长程结构连贯性，以及基于DOM增强的代理式流水线实现端到端功能验证。通过人工一致性分析验证自动评估协议的有效性。在单图与多图设置下对主流开源与专有VLMs的实验表明，网页长度增加导致结构保真度下降，且视觉上看似合理的生成结果往往无法支持可执行的多步交互。结果强调，评估长网页生成需超越视觉相似性，将可执行交互作为核心标准。代码与数据已开源。
原文摘要 · Abstract (English)
Recent vision-language models (VLMs) have shown promising progress in generating webpages from visual inputs, yet existing evaluations mainly focus on short, single-screen, and largely static webpages. We introduce LongWebBench, a benchmark for evaluating long-horizon webpage generation from both structural and functional perspectives. LongWebBench contains 490 real-world long webpages for structural fidelity evaluation and 507 goal-oriented interaction tasks over 129 webpages for functional evaluation. It employs two complementary protocols: a multi-dimensional VLM-based metric for assessing long-range structural coherence, and a DOM-augmented agent-based pipeline for end-to-end functional verification. We further examine the automatic evaluation protocols through human agreement analysis. Experiments with state-of-the-art open-source and proprietary VLMs under single-image and multi-image settings reveal that structural fidelity degrades as webpage length increases, while visually plausible generations often fail to support executable multi-step interactions. These results highlight the need to evaluate long webpage generation beyond visual similarity, with executable interaction as a core criterion. Our code and data are available at https://github.com/zheny2751-dotcom/LongWebBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。