用槽注意力引导扩散模型，实现更精准的物体发现与组合生成。
Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generation
- 通过适配器实现槽注意力对扩散模型的条件控制。
- 在多数据集上优于现有方法，真实图像生成效果显著提升。
- 无需外部监督，适合复杂现实场景的物体级生成任务。
我们提出SlotAdapt，一种将槽注意力与预训练扩散模型结合的物体中心学习方法，通过引入基于槽的适配器实现条件控制。该方法保留了预训练扩散模型的强大生成能力，同时避免其依赖文本的偏差。我们在架构中加入额外的引导损失，使适配器层的交叉注意力与槽注意力对齐，从而在不使用外部监督的情况下增强模型与输入图像中物体的一致性。实验表明，该方法在多个数据集（包括真实图像）上的物体发现与图像生成任务中均优于当前最优技术。进一步实验显示，相较于文献中其他基于槽的生成方法，我们的方法在复杂真实图像的组合生成任务中表现尤为出色。项目页面见 https://kaanakan.github.io/SlotAdapt/。
原文摘要 · Abstract (English)
We present SlotAdapt, an object-centric learning method that combines slot attention with pretrained diffusion models by introducing adapters for slot-based conditioning. Our method preserves the generative power of pretrained diffusion models, while avoiding their text-centric conditioning bias. We also incorporate an additional guidance loss into our architecture to align cross-attention from adapter layers with slot attention. This enhances the alignment of our model with the objects in the input image without using external supervision. Experimental results show that our method outperforms state-of-the-art techniques in object discovery and image generation tasks across multiple datasets, including those with real images. Furthermore, we demonstrate through experiments that our method performs remarkably well on complex real-world images for compositional generation, in contrast to other slot-based generative methods in the literature. The project page can be found at https://kaanakan.github.io/SlotAdapt/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。