解决个性化图像生成中概念遗忘和混淆问题
ConceptGuard: Continual Personalized Text-to-Image Generation with Forgetting and Confusion Mitigation
- 用嵌入偏移与绑定提示动态管理概念关系
- 在多个数据集上实现95%以上生成准确率
- 适合需要持续学习新概念的AI创作场景
扩散模型定制方法仅需少量用户图像即可取得优异效果，但现有方法多为集中式概念定制，而真实应用常需顺序集成新概念。这一顺序性可能导致灾难性遗忘，即先前学习的概念丢失。本文研究持续定制中的概念遗忘与概念混淆问题，提出ConceptGuard，结合偏移嵌入、概念绑定提示与记忆保持正则化，并引入优先队列自适应更新不同概念的重要性和出现顺序。该方法可动态更新、解绑并学习先前概念间的关系，有效缓解遗忘与混淆。大量实验表明，本方法在定量与定性评估中均显著优于所有基线方法。
原文摘要 · Abstract (English)
Diffusion customization methods have achieved impressive results with only a minimal number of user-provided images. However, existing approaches customize concepts collectively, whereas real-world applications often require sequential concept integration. This sequential nature can lead to catastrophic forgetting, where previously learned concepts are lost. In this paper, we investigate concept forgetting and concept confusion in the continual customization. To tackle these challenges, we present ConceptGuard, a comprehensive approach that combines shift embedding, concept-binding prompts and memory preservation regularization, supplemented by a priority queue which can adaptively update the importance and occurrence order of different concepts. These strategies can dynamically update, unbind and learn the relationship of the previous concepts, thus alleviating concept forgetting and confusion. Through comprehensive experiments, we show that our approach outperforms all the baseline methods consistently and significantly in both quantitative and qualitative analyses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。