无需训练即可实现多人物风格融合生成,解决图像拼贴与文本控制难题
A Training-Free Approach for Multi-ID Customization via Attention Adjustment and Spatial Control
- 通过解耦注意力机制注入不同身份嵌入,实现多身份图像生成
- 在IDBench基准上超越训练型方法,生成质量接近或更优
- 适合需要快速定制多角色图像的设计师与内容创作者
多身份定制是计算机视觉中的热门课题,旨在基于多个个体的图像生成融合其身份特征的定制化图像。相比单身份定制,该任务面临两大挑战:一是模型在推理时易出现图像复制粘贴问题,导致生成质量下降;二是文本控制能力弱,生成结果与输入文本描述不一致。本文提出无需训练的MultiID方法,利用现有单身份定制模型的优势,设计身份解耦的交叉注意力机制,将不同身份嵌入对应图像区域,生成多身份输出。同时引入局部提示、深度引导的空间控制和扩展自注意力三种策略,显著提升生成结果与文本提示及身份图像的一致性。我们构建了名为IDBench的评估基准,大量定性和定量实验表明,MultiID有效解决了上述两个问题,性能可媲美甚至超过基于训练的多身份定制方法。
原文摘要 · Abstract (English)
Multi-ID customization is an interesting topic in computer vision and attracts considerable attention recently. Given the ID images of multiple individuals, its purpose is to generate a customized image that seamlessly integrates them while preserving their respective identities. Compared to single-ID customization, multi-ID customization is much more difficult and poses two major challenges. First, since the multi-ID customization model is trained to reconstruct an image from the cropped person regions, it often encounters the copy-paste issue during inference, leading to lower quality. Second, the model also suffers from inferior text controllability. The generated result simply combines multiple persons into one image, regardless of whether it is aligned with the input text. In this work, we propose MultiID to tackle this challenging task in a training-free manner. Since the existing single-ID customization models have less copy-paste issue, our key idea is to adapt these models to achieve multi-ID customization. To this end, we present an ID-decoupled cross-attention mechanism, injecting distinct ID embeddings into the corresponding image regions and thus generating multi-ID outputs. To enhance the generation controllability, we introduce three critical strategies, namely the local prompt, depth-guided spatial control, and extended self-attention, making the results more consistent with the text prompts and ID images. We also carefully build a benchmark, called IDBench, for evaluation. The extensive qualitative and quantitative results demonstrate the effectiveness of MultiID in solving the aforementioned two challenges. Its performance is comparable or even better than the training-based multi-ID customization methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。