评测大模型生成网页应用的能力，聚焦视觉与功能一致性。
VISTA: An End-to-End Benchmark for Visual Spec-to-Web-App Coding Agents
- 设计五种不同信息条件的测试场景，覆盖从纯文本到带截图结构的输入。
- 引入交互组件标注与多维度评估，准确衡量网页的结构、行为和视觉匹配度。
- 发现视觉质量与功能正确性可分离，适合研究智能编程代理的开发者使用。
我们提出VISTA（视觉规范到网页应用基准），用于评估基于大模型的智能体在端到端网页应用生成方面的能力。与以往侧重算法任务的代码生成基准不同，VISTA聚焦真实场景下的以用户界面为中心的开发，要求智能体从不完整输入生成功能完整且视觉一致的应用。我们定义了五个提示信息条件，沿两个维度变化：视觉/结构保真度与技术栈约束。包括仅文本+自由选栈、文本+参考截图+三种指定栈、文本+参考截图+自由选栈、文本+截图+剪裁后的Figma结构+单一指定栈，以及文本+截图+剪裁后的Figma结构+自由选栈。为实现稳健评估，每个页面均人工标注了交互式UI组件及约三个视觉锚点，弥补了Playwright等脚本测试工具在开放生成任务中的局限性。评估结合基于DOM的参考匹配、行为特定浏览器测试和基于CLIP的视觉相似性，共同衡量结构对齐、行为完整性与整体视觉保真度。我们用VISTA评估了四个来自两种模型族和两种调用框架的智能体系统，发现视觉保真度与功能正确性在不同输入条件与智能体间存在部分解耦现象，且智能体编辑风格差异显著但基本与任务质量无关。VISTA为推进基于智能体的软件工程研究提供了严谨且可复现的基础。代码已开源：https://github.com/kaboider/VISTA_Bench。
原文摘要 · Abstract (English)
We present VISTA (VIsual Spec-To-App Benchmark), a benchmark for evaluating the end-to-end web-app generation capabilities of LLM-based agents. Unlike prior code generation benchmarks that focus on algorithmic tasks, VISTA targets realistic UI-centric development, where agents must produce functional, visually coherent applications from underspecified inputs. We define five prompt-information conditions that vary along two axes, visual/structural fidelity and stack constraint: (1) text only with free stack choice, (2) text with reference screenshots under three specified stacks, (3) text with reference screenshots under free stack choice, (4) text with screenshots and pruned Figma structure under a single specified stack, and (5) text with screenshots and pruned Figma structure under free stack choice. To enable robust evaluation, each page in the benchmark is manually annotated with interactive UI components and around three visual anchor points, addressing the well-known limitations of script-based testing tools such as Playwright in open-ended code generation settings. Evaluation combines DOM-grounded reference matching, behavior-specific browser tests, and CLIP-based visual similarity, jointly measuring structural alignment, behavioral completeness, and overall visual fidelity. We use VISTA to assess four agent systems drawn from two model families and two harnesses, finding that visual fidelity and functional correctness are partially decoupled across both input conditions and agents, and that agent editing style varies sharply but is largely orthogonal to task quality. VISTA establishes a rigorous and reproducible foundation for advancing agent-based software engineering research. Code is available at https://github.com/kaboider/VISTA_Bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。