让个性化图像生成不破坏原模型能力,实现纯净学习。
PureCC: Pure Learning for Text-to-Image Concept Customization
- 分离式学习目标,兼顾新概念与原模型行为。
- 在多个测试中保持原模型性能,同时实现高保真定制。
- 适合需要保留基础模型功能的图像生成应用。
现有概念定制方法在高保真度和多概念定制方面取得了显著成果,但常忽略学习新个性化概念时对原始模型行为与能力的影响。为此,我们提出 PureCC。PureCC 引入一种新型解耦学习目标,将目标概念的隐式引导与原始条件预测分离,使训练过程更聚焦于保持原模型特性。基于此目标,PureCC 设计双分支训练流程:冻结的提取器提供纯净的目标概念表示作为隐式引导,可训练的流模型生成原始条件预测,共同实现个性化概念的纯净学习。此外,PureCC 提出自适应引导尺度 $λ^ ext{⋆}$,动态调节目标概念引导强度,平衡定制保真度与模型保留。大量实验表明,PureCC 在保持原始行为与能力方面达到当前最优水平,同时实现高保真概念定制。代码已开源:https://github.com/lzc-sg/PureCC。
原文摘要 · Abstract (English)
Existing concept customization methods have achieved remarkable outcomes in high-fidelity and multi-concept customization. However, they often neglect the influence on the original model's behavior and capabilities when learning new personalized concepts. To address this issue, we propose PureCC. PureCC introduces a novel decoupled learning objective for concept customization, which combines the implicit guidance of the target concept with the original conditional prediction. This separated form enables PureCC to substantially focus on the original model during training. Moreover, based on this objective, PureCC designs a dual-branch training pipeline that includes a frozen extractor providing purified target concept representations as implicit guidance and a trainable flow model producing the original conditional prediction, jointly achieving pure learning for personalized concepts. Furthermore, PureCC introduces a novel adaptive guidance scale $λ^\star$ to dynamically adjust the guidance strength of the target concept, balancing customization fidelity and model preservation. Extensive experiments show that PureCC achieves state-of-the-art performance in preserving the original behavior and capabilities while enabling high-fidelity concept customization. The code is available at https://github.com/lzc-sg/PureCC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。