用视觉草图生成网页代码，提升准确性和效率。
Multimodal graph representation learning for website generation based on visual sketch
- 融合视觉与结构信息的多模态图学习方法
- 生成的HTML代码语义正确且结构合理，效果显著优于现有方法
- 适合需要自动化网页生成的研究者和开发者
Design2Code问题，即从数字设计自动生成可运行源码，因复杂性和耗时性成为软件开发中的重大挑战。传统方法难以准确解析网页设计中的复杂视觉细节和结构关系，导致自动化和效率受限。本文提出一种基于多模态图表示学习的新方法，整合设计草图的视觉与结构信息，显著提升代码生成的准确性和效率，尤其在生成语义正确、结构完整的HTML代码方面表现优异。通过全面评估，该方法在多项指标上显著优于现有技术，展现出革新设计到代码自动化的潜力。代码已公开于https://github.com/HySonLab/Design2Code。
原文摘要 · Abstract (English)
The Design2Code problem, which involves converting digital designs into functional source code, is a significant challenge in software development due to its complexity and time-consuming nature. Traditional approaches often struggle with accurately interpreting the intricate visual details and structural relationships inherent in webpage designs, leading to limitations in automation and efficiency. In this paper, we propose a novel method that leverages multimodal graph representation learning to address these challenges. By integrating both visual and structural information from design sketches, our approach enhances the accuracy and efficiency of code generation, particularly in producing semantically correct and structurally sound HTML code. We present a comprehensive evaluation of our method, demonstrating significant improvements in both accuracy and efficiency compared to existing techniques. Extensive evaluation demonstrates significant improvements of multimodal graph learning over existing techniques, highlighting the potential of our method to revolutionize design-to-code automation. Code available at https://github.com/HySonLab/Design2Code
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。