用自组织映射实现无需存储样本的持续学习,显著提升分类准确率。
Class Incremental Continual Learning with Self-Organizing Maps and Synthetic Replay
- 基于自组织映射构建统计内存,生成合成数据用于持续学习
- 在CIFAR-10和CIFAR-100上分别比现有最优无预训练编码器方法提升近10%和7%
- 无需存储原始数据,可可视化学习过程,适合资源受限场景
本文提出一种基于自组织映射(SOM)的生成式持续学习框架,通过引入学习到的分布统计量与编码器-解码器模型,实现类增量持续学习。该框架无需存储原始数据样本,仅保留每个SOM单元的均值、方差和协方差,用于生成合成样本进行回放。高维输入时,SOM在编码器-解码器的隐空间中运行;低维输入则独立运行。实验表明,该方法在标准类增量基准上性能媲美顶尖记忆型方法,并显著优于无记忆方法:在未使用预训练编码器的CIFAR-10和CIFAR-100上分别提升近10%和7%。使用基础编码器-解码器模型,在单类增量CIFAR-100上达到最佳表现,并首次提供TinyImageNet单类增量基线结果。方法支持训练后可视化与生成建模,为任务边界协议下的类增量持续学习提供了可扩展、标签无关、无需实例的解决方案。
原文摘要 · Abstract (English)
This work introduces a novel generative continual learning framework based on self-organizing maps (SOMs), a brain-inspired natural computing model, extended with learned distributional statistics and encoder--decoder models for class incremental continual learning. These extended SOMs enable exemplar-free replay with fixed-capacity statistical memory, eliminating the need to store raw data samples. For high-dimensional input spaces, the SOM operates over the latent space of the encoder--decoder, while for lower-dimensional inputs, the SOM operates in a standalone fashion. Our method stores a running mean, variance, and covariance for each SOM unit, from which synthetic samples are then generated during future learning iterations. For the encoder--decoder method, generated samples are fed through the decoder to be used in subsequent replay. Experimental results on standard class-incremental benchmarks show that our approach performs competitively with state-of-the-art memory-based methods and outperforms memory-free methods, notably improving over the best state-of-the-art single-class incremental performance without pretrained encoders on CIFAR-10 and CIFAR-100 by nearly 10% and 7%, respectively. We also achieve the best performance on single-class incremental CIFAR-100 using a foundational encoder-decoder and present the first baseline results for single-class incremental TinyImageNet. Our methodology facilitates easy visualization of the learning process and can also be utilized as a generative model post-training. Overall, the results demonstrate that SOM-based statistical replay offers a scalable, label-free, exemplar-free approach to class-incremental continual learning under standard task-boundary protocols.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。