解决扩散模型图文对齐不准问题，提升文本生成图像质量。
AI-T2I: Aggregating-and-Isolating Cross-Attention to Diffusion Models for Text-to-Image Synthesis
- 提出聚合与隔离交叉注意力机制，优化注意力分布。
- 在多个基准上超越当前最优方法，图文匹配更精准。
- 适用于布局控制和个性化生成，泛化能力强。
文本到图像合成得益于扩散模型强大的生成能力，但在去噪过程中，跨注意力图中的文本与图像对齐仍不精确。现有工作主要关注不同主体间注意力得分的重叠问题，忽略了相同主体内部注意力分散的问题。本文提出一种面向文本到图像合成的聚合与隔离交叉注意力方法（AI-T2I）。技术上，为解决注意力分散问题，设计了聚合损失，识别并整合散乱的内部注意力激活；在此基础上，引入隔离损失，将不同注意力之间的激活进一步分离，从而实现更精确的图文对齐。大量实验表明，AI-T2I 在多个基准上优于当前最优方法。此外，该方法在可控布局生成和个性化生成等任务中也展现出良好泛化性。代码已公开于 https://github.com/Hatter77/AI-T2I。
原文摘要 · Abstract (English)
Text-to-image synthesis has made significant progress, benefiting from the strong generative capabilities of diffusion models. However, these models struggle to achieve precise text-to-image alignment within cross-attention maps during the denoising process. Existing works primarily focus on inter-subject-token activations (i.e., cross-attention scores) overlap for different subjects, overlooking the intra-subject-token activations scattering issue for identical subjects. In this paper, we propose an Aggregating-and-Isolating cross-attention approach to diffusion models for Text-to-Image synthesis, dubbed AI-T2I. Technically, to address the scattering issue, we devise an aggregation loss to identify and consolidate the scattered intra-token activations, which implicitly helps mitigate the potential overlap issue. Upon that, an isolation loss is further introduced to push the inter-token activations apart, thus fulfilling precise text-to-image alignment. Extensive experiments on various benchmarks demonstrate the superiority of AI-T2I over the state-of-the-art works for text-to-image synthesis. Furthermore, our AI-T2I exhibits excellent generalization across other tasks, e.g., controllable layout generation and personalized generation. Our code is available at https://github.com/Hatter77/AI-T2I.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。