解决重叠布局生成图像时的错位问题，不需训练就能提升准确性。
ToLo: A Two-Stage, Training-Free Layout-To-Image Generation Framework For High-Overlap Layouts
- 分两阶段处理：先聚合再分离，用注意力图损失优化布局对齐。
- 在高重叠布局上显著减少属性泄露和漏画，性能超越现有方法。
- 适合需要精确布局控制的图像生成场景，如复杂插画设计。
近期无需训练的布局到图像扩散模型在生成可控布局的高质量图像方面表现优异。这些模型采用单阶段框架：通过基于注意力图的损失函数，引导每个概念的关注区域集中在对应区域内。然而，这些模型在处理存在显著重叠的布局时仍面临挑战，常导致属性泄露和实体缺失等问题。本文提出ToLo，一种针对高重叠布局的两阶段、无需训练的布局到图像生成框架。该框架包含两个阶段：聚合阶段与分离阶段，各自基于注意力图定义损失函数。为更有效评估，我们根据输入布局的交并比（IoU）对HRS数据集进行划分，构建了一个具有不同重叠程度的新数据集。在该数据集上的大量实验表明，ToLo在处理高重叠布局时显著提升了现有方法的性能。代码与数据集已公开：https://github.com/misaka12435/ToLo。
原文摘要 · Abstract (English)
Recent training-free layout-to-image diffusion models have demonstrated remarkable performance in generating high-quality images with controllable layouts. These models follow a one-stage framework: Encouraging the model to focus the attention map of each concept on its corresponding region by defining attention map-based losses. However, these models still struggle to accurately follow layouts with significant overlap, often leading to issues like attribute leakage and missing entities. In this paper, we propose ToLo, a two-stage, training-free layout-to-image generation framework for high-overlap layouts. Our framework consists of two stages: the aggregation stage and the separation stage, each with its own loss function based on the attention map. To provide a more effective evaluation, we partition the HRS dataset based on the Intersection over Union (IoU) of the input layouts, creating a new dataset for layout-to-image generation with varying levels of overlap. Through extensive experiments on this dataset, we demonstrate that ToLo significantly enhances the performance of existing methods when dealing with high-overlap layouts. Our code and dataset are available here: https://github.com/misaka12435/ToLo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。