用扩散模型特征实现少样本增量学习,无需存储图像即可避免遗忘。
Beyond Synthetic Replays: Turning Diffusion Features into Few-Shot Class-Incremental Learning Knowledge
- 从扩散模型中提取四类协同特征,统一处理新知识
- 在三个数据集上达到顶尖性能,参数量仅约600万
- 无需生成图像就能存取知识,适合资源受限场景
少样本增量学习(FSCIL)因训练数据极有限且需避免灾难性遗忘而极具挑战。现有方法多将稳定扩散(SD)用作生成回放数据的工具,但我们发现其多层次表征可作为统一主干网络。为此提出Diffusion-FSCIL:通过反演捕捉真实图像特征、以类别条件生成增强语义多样性、控制噪声注入提升泛化能力,并利用生成特征实现无图像存储的回放。与传统方法依赖合成缓冲区和独立分类主干不同,本框架完全在潜在空间运行,仅需轻量网络(约600万参数)。在CUB-200、miniImageNet和CIFAR-100上的实验表明性能达当前最优,全面消融验证了各特征必要性。此外,精简变体在保持竞争力的同时显著提升效率,证明生成模型可作为实用高效的FSCIL主干。
原文摘要 · Abstract (English)
Few-shot class-incremental learning (FSCIL) is challenging due to extremely limited training data while requiring models to acquire new knowledge without catastrophic forgetting. Recent works have explored generative models, particularly Stable Diffusion (SD), to address these challenges. However, existing approaches use SD mainly as a replay generator, whereas we demonstrate that SD's rich multi-scale representations can serve as a unified backbone. Motivated by this observation, we introduce Diffusion-FSCIL, which extracts four synergistic feature types from SD by capturing real image characteristics through inversion, providing semantic diversity via class-conditioned synthesis, enhancing generalization through controlled noise injection, and enabling replay without image storage through generative features. Unlike conventional approaches requiring synthetic buffers and separate classification backbones, our unified framework operates entirely in the latent space with only lightweight networks ($\approx$6M parameters). Extensive experiments on CUB-200, miniImageNet, and CIFAR-100 demonstrate state-of-the-art performance, with comprehensive ablations confirming the necessity of each feature type. Furthermore, we confirm that our streamlined variant maintains competitive accuracy while substantially improving efficiency, establishing the viability of generative models as practical and effective backbones for FSCIL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。