通过注意力引导提升文本图像生成中的属性绑定准确率
UNCAGE: Contrastive Attention Guidance for Masked Generative Transformers in Text-to-Image Generation
- 利用注意力图识别关键物体区域，优先解码相关词元
- 在多个基准上显著提升组合生成准确性，无额外推理开销
- 适用于希望提升生成细节对齐的视觉生成研究者
文本到图像（T2I）生成近年来广泛采用扩散模型和自回归模型。最近，掩码生成变压器因其双向注意力与并行解码，成为克服自回归模型因果注意力缺陷的替代方案，实现高效高质量生成。然而，组合式T2I生成仍具挑战，即使最先进的扩散模型也常无法准确绑定属性或实现正确图文对齐。尽管扩散模型对此问题已有大量研究，掩码生成变压器虽存在类似局限却未被充分探索。为此，我们提出无需训练的新型方法UNCAGE（Unmasking with Contrastive Attention Guidance），通过注意力图引导优先解码明确代表个体对象的词元，从而提升组合保真度。UNCAGE在多个基准和指标上均实现定量与定性性能提升，且推理开销可忽略不计。代码已开源：https://github.com/furiosa-ai/uncage。
原文摘要 · Abstract (English)
Text-to-image (T2I) generation has been actively studied using Diffusion Models and Autoregressive Models. Recently, Masked Generative Transformers have gained attention as an alternative to Autoregressive Models to overcome the inherent limitations of causal attention and autoregressive decoding through bidirectional attention and parallel decoding, enabling efficient and high-quality image generation. However, compositional T2I generation remains challenging, as even state-of-the-art Diffusion Models often fail to accurately bind attributes and achieve proper text-image alignment. While Diffusion Models have been extensively studied for this issue, Masked Generative Transformers exhibit similar limitations but have not been explored in this context. To address this, we propose Unmasking with Contrastive Attention Guidance (UNCAGE), a novel training-free method that improves compositional fidelity by leveraging attention maps to prioritize the unmasking of tokens that clearly represent individual objects. UNCAGE consistently improves performance in both quantitative and qualitative evaluations across multiple benchmarks and metrics, with negligible inference overhead. Our code is available at https://github.com/furiosa-ai/uncage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。