无需标注数据,自动精准编辑设计布局并保持结构完整。
ReLayout: Versatile and Structure-Preserving Design Layout Editing via Relation-Aware Design Reconstruction
- 构建关系图约束未编辑元素的布局结构,确保修改不破坏原有排布。
- 通过自监督重建机制模拟编辑过程,解决缺乏三元组数据难题。
- 统一多类编辑动作,支持自然语言指令下的灵活设计调整。
自动化重设计无需人工干预,是设计流程的重要进步。本文聚焦基础任务——设计布局编辑,旨在根据用户意图自主修改设计的几何构成。为应对自然语言表达中用户需求的模糊性,我们定义了四种基本且关键的编辑操作,并标准化其格式。该任务面临双重挑战:在满足指定编辑操作的同时,保持未编辑元素的布局结构;以及缺乏(原始设计、编辑操作、编辑后设计)三元组样本。为此,我们提出ReLayout框架,实现无需三元组数据的多样化且结构保留的布局编辑。ReLayout首先引入关系图,包含未编辑元素间的相对位置与尺寸关系,作为布局结构保持的约束。随后提出关系感知的设计重建(RADR),通过学习从元素、关系图和合成编辑操作中重建设计,以自监督方式有效模拟编辑过程。多模态大语言模型作为RADR骨干,将多种编辑动作统一于单一模型中,微调后即可实现多样化编辑。定性、定量结果及用户研究显示,ReLayout在编辑质量、准确性和布局结构保持方面显著优于基线模型。
原文摘要 · Abstract (English)
Automated redesign without manual adjustments marks a key step forward in the design workflow. In this work, we focus on a foundational redesign task termed design layout editing, which seeks to autonomously modify the geometric composition of a design based on user intents. To overcome the ambiguity of user needs expressed in natural language, we introduce four basic and important editing actions and standardize the format of editing operations. The underexplored task presents a unique challenge: satisfying specified editing operations while simultaneously preserving the layout structure of unedited elements. Besides, the scarcity of triplet (original design, editing operation, edited design) samples poses another formidable challenge. To this end, we present ReLayout, a novel framework for versatile and structure-preserving design layout editing that operates without triplet data. Specifically, ReLayout first introduces the relation graph, which contains the position and size relationships among unedited elements, as the constraint for layout structure preservation. Then, relation-aware design reconstruction (RADR) is proposed to bypass the data challenge. By learning to reconstruct a design from its elements, a relation graph, and a synthesized editing operation, RADR effectively emulates the editing process in a self-supervised manner. A multi-modal large language model serves as the backbone for RADR, unifying multiple editing actions within a single model and thus achieving versatile editing after fine-tuning. Qualitative, quantitative results and user studies show that ReLayout significantly outperforms the baseline models in terms of editing quality, accuracy, and layout structure preservation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。