无需重新训练，一键合并多个个性化图像模型。
LoRACLR: Contrastive Adaptation for Customization of Diffusion Models
- 用对比学习对齐多个LoRA模型的权重空间。
- 可融合多个概念，生成高质量多主题图像。
- 适合需要快速组合个性化模型的用户。
文本到图像的个性化生成技术已能高保真地生成包含特定概念的图像，但现有方法在整合多个个性化模型时易出现属性混淆，或需分别训练以保持概念独立性。本文提出LoRACLR，一种新型多概念图像生成方法，通过对比目标将多个针对不同概念微调的LoRA模型合并为单一统一模型，无需额外单独微调。该方法对齐并融合各模型的权重空间，确保兼容性同时最小化干扰。通过为每个概念建立清晰且一致的表征，LoRACLR实现了高效、可扩展的模型组合，支持高质量多概念图像合成。实验表明，该方法能准确融合多个概念，显著提升个性化图像生成能力。
原文摘要 · Abstract (English)
Recent advances in text-to-image customization have enabled high-fidelity, context-rich generation of personalized images, allowing specific concepts to appear in a variety of scenarios. However, current methods struggle with combining multiple personalized models, often leading to attribute entanglement or requiring separate training to preserve concept distinctiveness. We present LoRACLR, a novel approach for multi-concept image generation that merges multiple LoRA models, each fine-tuned for a distinct concept, into a single, unified model without additional individual fine-tuning. LoRACLR uses a contrastive objective to align and merge the weight spaces of these models, ensuring compatibility while minimizing interference. By enforcing distinct yet cohesive representations for each concept, LoRACLR enables efficient, scalable model composition for high-quality, multi-concept image synthesis. Our results highlight the effectiveness of LoRACLR in accurately merging multiple concepts, advancing the capabilities of personalized image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。