通过注意力分布迁移实现图像主体定制，提升生成一致性与语义准确度。
Redirecting the Flow: Image Customization through Attention Distribution Shift
- 基于最大熵理论构建注意力分布迁移框架，实现参考图像与生成过程的对齐。
- 在DreamBooth和Custom101上优于现有方法，兼顾语义忠实度与主体一致性。
- 适合需要高保真主体复现的图像生成场景，如个性化设计、虚拟形象生成。
以主体为导向的图像定制旨在生成既符合文本指令又保留参考主体身份的图像。现有方法包括测试时微调、编码器驱动方法以及共享注意力空间中的标记竞争，存在效率有限、参考特征提取与生成过程错位、无关信息干扰等问题。为解决这些局限，本文将定制任务建模为引入参考图像后文本到图像生成中的分布偏移，并基于最大熵理论推导出条件注意力分布偏移公式。在此基础上，提出CustomShift——一种基于Stable Diffusion 3的双分支架构。参考对齐分支利用参考图像与主体名称之间的自注意力实现逐层与潜在表示的对齐；跨引导分支融合文本与参考线索以指导生成。在DreamBooth与Custom101基准上的实验表明，该方法持续优于当前最先进方法，在语义保真度与主体一致性之间取得更优平衡。
原文摘要 · Abstract (English)
Subject-driven image customization aims to generate images that not only follow textual instructions but also preserve the identity of a given reference subject. Existing approaches, including test-time fine-tuning, encoder-based methods, and token competition in shared attention spaces, suffer from limited efficiency, misalignment between extracted reference features and the generative process, and interference from irrelevant information. To address these limitations, we formulate the customization task as a distribution shift induced by incorporating reference images into text-to-image generation, and derive a Conditional Attention Distribution Shift formulation grounded in maximum entropy theory. Building on this formulation, we propose CustomShift, a dual-branch architecture based on Stable Diffusion 3. The Reference-Alignment Branch leverages self-attention between reference images and subject names to achieve layer-wise alignment with latent representations, while the Cross-Guidance Branch integrates textual and reference cues to guide generation. Experiments on the DreamBooth and Custom101 benchmarks demonstrate that our method consistently outperforms state-of-the-art approaches, achieving a better balance between semantic fidelity and subject consistency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。