先分离概念再生成，解决多概念图像生成错漏问题
Rectify Then Diffuse: Disentangling Concepts Before Denoising Trajectory Unfolds
- 用可微分方法在去噪前重排概念位置，避免注意力耦合
- 在AE-Bench上比CO3提升BLIP-VQA 45.8%、ImageReward 19.6%
- 无需训练，速度更快，适合需要高精度组合生成的场景
文本到图像扩散模型能良好生成单个概念，但在多概念组合时常出现遗漏或混淆。我们发现失败根源在于去噪前的早期协调瓶颈：提示条件注意力可能将不同概念分配到高度重叠的空间区域，导致去噪过程中注意力持续耦合。这促使我们将组合生成视为边界条件问题，而非反复控制演化轨迹。为此，我们提出无需训练的Rectify-then-Diffuse（RTD）框架，在标准去噪前一次性修正初始分配。首先，提出软重叠解耦（SOD），将初始概念图的归一化重叠转化为可微且布局无关的分离目标；其次，引入各向同性梯度校正（IGR），对SOD梯度进行归一化，并施加尺度一致的有界潜在位移。大量实验表明，RTD在组合保真度上达到当前最优，且显著提升鲁棒性。在AE-Bench对象对子集上，相比CO3，RTD使BLIP-VQA提升45.8%，ImageReward提升19.6%，同时运行速度提升2.3倍。代码将发布于https://github.com/Z-yiwei/rectify-then-diffuse
原文摘要 · Abstract (English)
Text-to-image diffusion models can generate individual concepts well, but they often omit or merge concepts incorrectly with multiple concepts. We trace these failures to an early coordination bottleneck: before denoising begins, prompt-conditioned attention may allocate different concepts to strongly overlapping spatial support, which can keep their attention coupled as denoising proceeds. This observation motivates treating compositional generation as a boundary-condition problem rather than repeatedly controlling the evolving trajectory. To this end, we propose Rectify-then-Diffuse (RTD), a training-free framework that rectifies the initial allocation once before standard denoising. Firstly, we propose Soft-Overlap Disentanglement (SOD), which converts normalized overlap between pilot concept maps into a differentiable and layout-agnostic separation objective. Secondly, we introduce Isotropic Gradient Rectification (IGR), which normalizes the SOD gradient and applies a bounded latent displacement with a consistent scale across prompts and initializations. Extensive experiments show that RTD achieves state-of-the-art compositional fidelity and robust gains. On the AE-Bench object pair subset, RTD improves BLIP-VQA by 45.8% and ImageReward by 19.6% over CO3 while running 2.3$\times$ faster. Code will be released at https://github.com/Z-yiwei/rectify-then-diffuse
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。