用扩散模型替代梯度更新,实现零训练的少样本增量学习。
Breaking Forgetting: Training-Free Few-Shot Class-Incremental Learning via Conditional Diffusion
- 用条件扩散过程替代梯度优化,无需训练即可增量学习新类别。
- 在多个主流基准上达到最优性能,且计算与内存开销大幅降低。
- 结合大模型生成的文本描述,缓解少样本下的表征不足问题。
为克服少样本增量学习(FSCIL)中的灾难性遗忘问题,现有工作多聚焦于改进基于梯度的优化策略,却忽视了随着新类别增多带来的训练成本爆炸式增长。尤其在每类仅提供少量样本的情况下,梯度更新不仅加剧对基础类别的遗忘,也阻碍新类别的适应。本文提出一个无需训练的FSCIL范式,通过揭示梯度优化与条件扩散过程之间的内在联系,构建基于扩散的生成迁移机制(CD-FSCIL),完全摒弃梯度更新。为进一步提升少样本下的表征能力,引入融合视觉特征与大语言模型自动生成的自然语言描述的多模态学习策略,显著缓解样本稀缺问题并增强泛化能力。大量实验表明,该方法在主流FSCIL基准上均达领先性能,同时大幅降低计算与内存开销,推动持续适应向零训练范式演进。
原文摘要 · Abstract (English)
Efforts to overcome catastrophic forgetting in Few-Shot Class-Incremental Learning (FSCIL) have primarily focused on developing more effective gradient-based optimization strategies. In contrast, little attention has been paid to the training cost explosion that inevitably arises as the number of novel classes increases, a consequence of relying on gradient learning even under extreme data scarcity. More critically, since FSCIL typically provides only a few samples for each new class, gradient-based updates not only induce severe catastrophic forgetting on base classes but also hinder adaptation to novel ones. This paper seeks to break this long-standing limitation by asking: Can we design a training-free FSCIL paradigm that entirely removes gradient optimization? We provide an affirmative answer by uncovering an intriguing connection between gradient-based optimization and the Conditional Diffusion process. Building on this observation, we propose a Conditional Diffusion-driven FSCIL (CD-FSCIL) framework that substitutes the conventional gradient update process with a diffusion-based generative transition, enabling training-free incremental adaptation while effectively mitigating forgetting. Furthermore, to enhance representation under few-shot constraints, we introduce a multimodal learning strategy that integrates visual features with natural language descriptions automatically generated by Large Language Models (LLMs). This synergy substantially alleviates the sample scarcity issue and improves generalization across novel classes. Extensive experiments on mainstream FSCIL benchmarks demonstrate that our method not only achieves state-of-the-art performance but also drastically reduces computational and memory overhead, marking a paradigm shift toward training-free continual adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。