将UI设计转化为带资源依赖的多页网页，提升真实场景生成能力
MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs
- 引入资源列表结构追踪页面资源与链接关系
- 资源列表使导航功能从0%提升至66%-80%
- 适用于需要真实多页网站生成的研究与开发
多页网站主导现代网页开发，但现有设计转代码方法受限于单页、自包含假设，无法处理外部资源连接。为此，我们提出多页资源感知网页（MRWeb）生成任务，将UI设计转化为具备内部/外部导航、图像加载与后端路由的多页可运行网页。我们设计了新的资源列表数据结构以追踪资源、链接和组件。基于新构建的500个网站数据集（300个合成，200个真实世界），我们评估现有方法在该任务上的表现，分析资源列表对生成效果的影响，研究多模态大模型（MLLM）的局限性，并验证工具在真实工作流中的有效性。结果表明，资源列表使导航功能从0%提升至66%-80%，同时改善视觉相似性。我们提出的评估指标与框架为理解MLLM在MRWeb任务中的性能提供了新视角。相关工具、数据集与评估框架已开源。
原文摘要 · Abstract (English)
Multi-page websites dominate modern web development. However, existing design-to-code methods rely on simplified assumptions, limiting to single-page, self-contained webpages without external resource connection. To address this gap, we introduce the Multi-Page Resource-Aware Webpage (MRWeb) generation task, which transforms UI designs into multi-page, functional web UIs with internal/external navigation, image loading, and backend routing. We propose a novel resource list data structure to track resources, links, and design components. Our study applies existing methods to the MRWeb problem using a newly curated dataset of 500 websites (300 synthetic, 200 real-world). Specifically, we identify the best metric to evaluate the similarity of the web UI, assess the impact of the resource list on MRWeb generation, analyze MLLM limitations, and evaluate the effectiveness of the MRWeb tool in real-world workflows. The results show that resource lists boost navigation functionality from 0% to 66%-80% while facilitating visual similarity. Our proposed metrics and evaluation framework provide new insights into MLLM performance on MRWeb tasks. We release the MRWeb tool, dataset, and evaluation framework to promote further research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。