首个综合评估多模态大模型前端代码生成的基准，覆盖三大主流框架
DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generation
- 构建涵盖React/Vue/Angular的多框架评估体系，模拟真实开发流程
- 包含900个网页样本、9类编辑任务与6类错误修复，支持多维度分析
- 揭示模型在不同框架、任务和输入条件下的性能瓶颈，指导未来研究
多模态大语言模型在自动化前端工程中展现出强大能力，如从视觉设计生成UI代码。然而现有前端代码生成基准存在三方面局限：(1) 当前主流开发框架（如React、Vue、Angular）未被纳入；(2) 评估仅聚焦代码生成，忽略实际开发中反复修改与修复的迭代过程；(3) 评价方式单一，缺乏对任务难度、输入上下文变化及代码级细节的深入分析。为此，我们提出DesignBench，一个面向MLLM的多框架、多任务综合评估基准。该基准涵盖三种主流前端框架（React、Vue、Angular）及原生HTML/CSS，评估生成、编辑、修复三类核心前端任务，覆盖真实开发流程。DesignBench包含900个网页样本，涉及11个主题、9种编辑类型和6类问题类别，支持多维度性能分析。系统性评估揭示了模型在不同框架下的局限性、任务相关的瓶颈以及在不同条件下的表现差异，为未来自动化前端开发研究提供重要指导。代码与数据已开源：https://github.com/WebPAI/DesignBench。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in automated front-end engineering, e.g., generating UI code from visual designs. However, existing front-end UI code generation benchmarks have the following limitations: (1) While framework-based development becomes predominant in modern front-end programming, current benchmarks fail to incorporate mainstream development frameworks. (2) Existing evaluations focus solely on the UI code generation task, whereas practical UI development involves several iterations, including refining editing, and repairing issues. (3) Current benchmarks employ unidimensional evaluation, lacking investigation into influencing factors like task difficulty, input context variations, and in-depth code-level analysis. To bridge these gaps, we introduce DesignBench, a multi-framework, multi-task evaluation benchmark for assessing MLLMs' capabilities in automated front-end engineering. DesignBench encompasses three widely-used UI frameworks (React, Vue, and Angular) alongside vanilla HTML/CSS, and evaluates on three essential front-end tasks (generation, edit, and repair) in real-world development workflows. DesignBench contains 900 webpage samples spanning over 11 topics, 9 edit types, and 6 issue categories, enabling detailed analysis of MLLM performance across multiple dimensions. Our systematic evaluation reveals critical insights into MLLMs' framework-specific limitations, task-related bottlenecks, and performance variations under different conditions, providing guidance for future research in automated front-end development. Our code and data are available at https://github.com/WebPAI/DesignBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。