解决多参考图生成时身份混淆问题，提升图像定制的保真与可扩展性。
UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward
- 采用全局匹配优化框架，通过强化学习实现多对多身份一致
- 在多个定制化方法上显著提升身份一致性并降低混淆度
- 适用于需要高保真身份保留的图像生成场景，如虚拟形象设计
近期图像定制技术因更强的个性化能力展现出广泛应用前景。然而，由于人类对人脸更为敏感，使用多参考图像时仍面临身份保持一致与避免混淆的挑战，限制了定制模型的身份可扩展性。为此，我们提出UMO——统一多身份优化框架，旨在维持高保真身份一致性并提升可扩展性。UMO采用“多对多匹配”范式，将多身份生成重构为全局分配优化问题，并通过扩散模型上的强化学习，普遍增强现有图像定制方法的多身份一致性。为支持UMO训练，我们构建了一个包含合成与真实数据的可扩展定制数据集。此外，我们提出一种新指标用于衡量身份混淆。大量实验表明，UMO不仅显著提升身份一致性，还在多个图像定制方法上有效降低身份混淆，在开源方法中达到身份保持维度的新最佳水平。
原文摘要 · Abstract (English)
Recent advancements in image customization exhibit a wide range of application prospects due to stronger customization capabilities. However, since we humans are more sensitive to faces, a significant challenge remains in preserving consistent identity while avoiding identity confusion with multi-reference images, limiting the identity scalability of customization models. To address this, we present UMO, a Unified Multi-identity Optimization framework, designed to maintain high-fidelity identity preservation and alleviate identity confusion with scalability. With "multi-to-multi matching" paradigm, UMO reformulates multi-identity generation as a global assignment optimization problem and unleashes multi-identity consistency for existing image customization methods generally through reinforcement learning on diffusion models. To facilitate the training of UMO, we develop a scalable customization dataset with multi-reference images, consisting of both synthesised and real parts. Additionally, we propose a new metric to measure identity confusion. Extensive experiments demonstrate that UMO not only improves identity consistency significantly, but also reduces identity confusion on several image customization methods, setting a new state-of-the-art among open-source methods along the dimension of identity preserving. Code and model: https://github.com/bytedance/UMO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。