用检索模板引导生成,让版面设计更符合条件约束。
LayoutRAG: Retrieval-Augmented Model for Content-agnostic Conditional Layout Generation
- 根据条件检索相似版面模板作为参考。
- 生成过程受检索模板指导,布局质量显著提升。
- 适合需要精准控制版面结构的设计场景。
可控版面生成旨在根据特定约束(如元素类型或位置)生成合理的图形设计布局。尽管近期基于扩散或流匹配的模型在多种条件生成任务中取得进展,但在给定条件下生成最优布局方面仍有改进空间。本文提出通过条件检索与参考引导生成相结合的方法:首先根据给定条件检索合适的布局模板作为参考,再利用这些模板指导去噪或基于流的生成过程。通过检索与条件兼容的布局,模型能捕捉到条件中未明确提供的潜在信息,从而在生成过程中获得更有效的引导。相比以往直接将条件输入模型并由其推断缺失属性的方式,本方法更具优势。此外,我们设计了条件调制注意力机制,可选择性地吸收检索知识,适应检索模板与条件之间的差异。大量实验表明,该方法能有效生成符合要求的高质量布局,并优于现有最先进模型。代码将在论文接受后公开。
原文摘要 · Abstract (English)
Controllable layout generation aims to create plausible visual arrangements of element bounding boxes within a graphic design according to certain optional constraints, such as the type or position of a specific component. While recent diffusion or flow-matching models have achieved considerable advances in multifarious conditional generation tasks, there remains considerable room for generating optimal arrangements under given conditions. In this work, we propose to carry out layout generation through retrieving by conditions and reference-guided generation. Specifically, we retrieve appropriate layout templates according to given conditions as references. The references are then utilized to guide the denoising or flow-based transport process. By retrieving layouts compatible with the given conditions, we can uncover the potential information not explicitly provided in the given condition. Such an approach offers more effective guidance to the model during the generation process, in contrast to previous models that feed the condition to the model and let the model infer the unprovided layout attributes directly. Meanwhile, we design a condition-modulated attention that selectively absorbs retrieval knowledge, adapting to the difference between retrieved templates and given conditions. Extensive experiment results show that our method successfully produces high-quality layouts that meet the given conditions and outperforms existing state-of-the-art models. Code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。