arXiv:2409.16689cs.CVcs.AI2024-09ECCV被引 8

提出布局纠错模块,解决离散扩散模型生成布局后难以修正的问题。

Layout-Corrector: Alleviating Layout Sticking Phenomenon in Discrete Diffusion Model

  • 引入布局评估模块,动态识别不和谐元素并重置
  • 在多个基准上提升生成效果,尤其改善快速采样时性能下降
  • 适合需要高质量布局生成的设计师与自动化工具

布局生成旨在合成包含类别、位置、尺寸等属性元素的协调布局。尽管人类设计师可通过调整元素位置来优化美感,我们发现当前离散扩散模型(DDMs)在生成后难以修正不协调布局。本文首次揭示了DDMs中布局粘滞现象的本质,并提出一种简单有效的布局评估模块Layout-Corrector,可与现有DDMs协同工作以缓解该问题。该模块基于学习,能识别布局中不和谐的元素,综合考虑整体构图复杂性。生成过程中,Layout-Corrector评估每个标记的合理性,将低分标记重置为未生成状态,使DDM以高分标记为线索重新生成更协调的元素。在常见基准上的测试表明,该模块在与多种先进DDMs结合时持续提升布局生成性能。大量分析显示:(1) 能准确识别错误标记;(2) 可调控保真度-多样性权衡;(3) 显著缓解快速采样导致的性能下降。

原文摘要 · Abstract (English)

Layout generation is a task to synthesize a harmonious layout with elements characterized by attributes such as category, position, and size. Human designers experiment with the placement and modification of elements to create aesthetic layouts, however, we observed that current discrete diffusion models (DDMs) struggle to correct inharmonious layouts after they have been generated. In this paper, we first provide novel insights into layout sticking phenomenon in DDMs and then propose a simple yet effective layout-assessment module Layout-Corrector, which works in conjunction with existing DDMs to address the layout sticking problem. We present a learning-based module capable of identifying inharmonious elements within layouts, considering overall layout harmony characterized by complex composition. During the generation process, Layout-Corrector evaluates the correctness of each token in the generated layout, reinitializing those with low scores to the ungenerated state. The DDM then uses the high-scored tokens as clues to regenerate the harmonized tokens. Layout-Corrector, tested on common benchmarks, consistently boosts layout-generation performance when in conjunction with various state-of-the-art DDMs. Furthermore, our extensive analysis demonstrates that the Layout-Corrector (1) successfully identifies erroneous tokens, (2) facilitates control over the fidelity-diversity trade-off, and (3) significantly mitigates the performance drop associated with fast sampling.

布局生成扩散模型图像编辑自动化设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。