拆解界面结构与渲染，提升设计转代码的精度与效率
DOne: Decoupling Structure and Rendering for High-Fidelity Design-to-Code Generation
- 分两步生成：先解析布局结构，再精细还原元素
- 在高复杂度界面中实现超过10%的视觉相似度提升
- 适合需要高保真界面生成的开发与设计团队
尽管视觉语言模型（VLMs）在设计转代码任务中展现出潜力，但仍存在‘整体性瓶颈’——难以兼顾高层级结构与细粒度视觉细节，常导致布局失真或使用通用占位符。为此，我们提出 DOne，一种端到端框架，将结构理解与元素渲染解耦。DOne 引入三部分创新：(1) 学习型布局分割模块，可分解复杂设计，避免启发式裁剪的局限；(2) 专用混合元素检索器，有效处理用户界面组件的极端长宽比与高密度；(3) 基于模式引导的生成范式，连接布局与代码。为严格评估性能，我们构建了 HiFi2Code 基准，其布局复杂度显著高于现有数据集。在 HiFi2Code 上的大量实验表明，DOne 在高层视觉相似度（如 GPT Score 超过10%）和细粒度元素对齐方面均优于现有方法。人工评估证实其可带来三倍生产力提升，并保持更高视觉保真度。
原文摘要 · Abstract (English)
While Vision Language Models (VLMs) have shown promise in Design-to-Code generation, they suffer from a "holistic bottleneck-failing to reconcile high-level structural hierarchy with fine-grained visual details, often resulting in layout distortions or generic placeholders. To bridge this gap, we propose DOne, an end-to-end framework that decouples structure understanding from element rendering. DOne introduces (1) a learned layout segmentation module to decompose complex designs, avoiding the limitations of heuristic cropping; (2) a specialized hybrid element retriever to handle the extreme aspect ratios and densities of UI components; and (3) a schema-guided generation paradigm that bridges layout and code. To rigorously assess performance, we introduce HiFi2Code, a benchmark featuring significantly higher layout complexity than existing datasets. Extensive evaluations on the HiFi2Code demonstrate that DOne outperforms exiting methods in both high-level visual similarity (e.g., over 10% in GPT Score) and fine-grained element alignment. Human evaluations confirm a 3 times productivity gain with higher visual fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。