arXiv:2606.04797cs.CVcs.LG2026-06TPAMI

让扩散模型持续学习新概念,不遗忘旧概念。

Crafting Your Evolving Dreams: Concept-Incremental Versatile Customization

论文配图:Crafting Your Evolving Dreams: Concept-Incremental Versatile Customization
图 1 · 摘自论文原文
  • 用解耦属性的LoRA模块和相关性引导聚合策略
  • 在连续学习中减少遗忘,提升多概念合成一致性
  • 适合需要不断添加新个性化概念的用户

定制化扩散模型(CDMs)因其生成个性化概念的强大能力受到广泛关注。然而,现有方法通常假设用户的概念集合是静态的,无法随时间增量扩展。此外,在连续学习新概念时,它们容易出现灾难性遗忘和对已学概念的忽视。为解决上述问题,我们提出一种可持续定制的扩散模型(CCDM),支持概念增量式多样化定制。具体地,设计了属性解耦的LoRA(AD-LoRA)模块与相关性引导的AD-LoRA聚合策略,以保留各任务特有的概念属性,并利用任务间的有益关联增强新任务的持续学习能力。为应对概念忽视问题,提出可控区域上下文合成策略,实现符合用户条件的多概念组合,通过保证用户定义区域间的语义独立性和平滑边界过渡,提升多概念合成的整体一致性。实验表明,我们的CCDM在多个基准上显著优于基线方法。

原文摘要 · Abstract (English)

Custom diffusion models (CDMs) have garnered significant interest owing to their remarkable capacity for generating personalized concepts. However, the majority of CDMs unrealistically presume that the user's collection of personalized concepts is static and incapable of incremental growth over time. Furthermore, they exhibit significant catastrophic forgetting and concept neglect of previously learned concepts when incrementally learning a sequence of new ones. To resolve the above challenges, we develop a novel Continually Customizable Diffusion Model (CCDM), enabling users to perform concept-incremental versatile customization. Specifically, we design an attribute-decoupled LoRA (AD-LoRA) module and a relevance-guided AD-LoRA aggregation strategy to mitigate catastrophic forgetting. They can preserve concept-specific attributes of each task and leverage beneficial inter-task correlations to enhance the continual learning of new customization tasks. Additionally, to address the challenge of concept neglect, we propose a controllable regional context synthesis strategy that performs multi-concept composition in alignment with user-provided conditions. This strategy enhances the overall consistency in multi-concept synthesis by guaranteeing semantic independence between user-defined regions and their smooth boundary transitions. Experiments show our CCDM exhibits significant improvements over baseline methods.

扩散模型持续学习个性化生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。