arXiv:2606.00154cs.SEcs.AI2026-06被引 2

首个评估交互式网页代码生成的基准,覆盖871个真实交互动作。

Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages

论文配图:Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages
图 1 · 摘自论文原文
  • 构建包含复杂交互路径的真实网页数据集
  • 发现现有模型在交互一致性上表现显著不足
  • 适合前端开发与多模态模型研究者使用

多模态大模型在视觉推理与代码生成方面取得进展,可将界面设计直接转为可执行代码,提升前端开发效率。但现有基准多聚焦静态网页,忽略真实应用中的动态交互行为。且评价标准仅关注视觉一致性和代码结构,忽视生成页面与参考页面间的交互一致性。为此,本文提出WebIGBench,首个针对复杂交互网页代码生成的评估基准。通过人工设计交互路径与UI自动化,从真实网站收集了103个复杂网页,涵盖5类常见交互(如点击、输入),共871个独立交互动作。同时提出新的自动化评估流程,填补交互行为评估空白。对多个代表性多模态大模型的实验揭示了当前模型在交互生成上的性能边界。该基准已开源:https://github.com/anoa12159-hue/WebIGBench_eval。

原文摘要 · Abstract (English)

Recent advancements in multimodal large language models (MLLMs) have achieved remarkable progress in multimodal reasoning and code generation, catalyzing a new paradigm for front-end development. In particular, these models can directly transform visual designs into executable code, significantly improving the efficiency and adaptability of web development. Modern web applications are dynamic and interactive, featuring frequent user-page interactions. However, existing benchmarks largely evaluate the code generation of static webpages, ignoring the complex interactive behaviors in real-world applications. Besides, their evaluation criteria remain confined to visual fidelity and code structure, overlooking the interaction consistency between the generated and the reference webpages. To address these limitations, we introduce WebIGBench, the first benchmark designed to evaluate code generation for interactive webpages with complex interactions. By combining manually designed interaction paths with UI automation, we collected 103 complex webpages from real-world websites. This benchmark covers 5 popular interactive action types (e.g., click, input) involving 871 distinct interactive actions. Moreover, we propose a novel evaluation pipeline to address the gap in automated assessment of interactive actions. Extensive experiments on several representative MLLMs reveal the performance boundaries of current models in interactive webpage code generation using WebIGBench. The proposed benchmark is available at https://github.com/anoa12159-hue/WebIGBench_eval.

多模态模型代码生成交互评估前端开发

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。