无需训练图像特征,用随机水库特征实现高效长时冷启动持续学习
Data-Free Reservoir Features for Efficient Long-Horizon Cold-Start Continual Learning
- 采用固定二维水库特征+流式线性判别分析头,不需回放或反向传播
- 在500任务分割下性能显著优于现有方法,训练速度远超传统方法
- 适合资源受限场景下的长期增量学习,尤其冷启动阶段无先验数据
冷启动无样本类增量学习要求在不使用回放、外部预训练或大规模初始任务的情况下持续学习新增类别。现有方法要么全程训练主干网络以补偿语义漂移,要么冻结主干导致初期类别偏差。前者计算开销随任务增长而上升,后者在冷启动下表现弱。本文提出CIRCLE:一种基于固定双向二维水库特征(BiRC2D)的分类器,通过流式闭式更新的线性判别分析头实现增量学习。将多个随机水库实例组合为特征集成,并平均多个SLDA头的softmax输出,实现可调偏差-方差权衡。特征提取器不参与训练,头部支持流式更新,整个过程无需样本回放、任务边界信息或主干反向传播。在CIFAR-100、TinyImageNet、ImageNet-Subset和ImageNet-1k上,CIRCLE在10–20任务分割下具有竞争力,在50、100和500任务分割下显著优于强基线方法,且训练速度远快于需重复训练主干的方法。消融实验表明,BiRC2D特征提取器、SLDA头部及特征/预测集成策略均对性能有贡献。
原文摘要 · Abstract (English)
Cold-start exemplar-free class-incremental learning requires learning a growing set of classes without replay, external pretraining, or a large initial task. Existing cold-start methods typically either train the backbone throughout the stream and compensate for semantic drift, or freeze a backbone after the first task, producing features biased toward the initial classes. These choices also create a computational tension: drift-compensation methods require repeated backbone training and increasingly expensive updates as the task horizon grows, while frozen-backbone methods are cheap but weak under cold start. We study a third option: a feature extractor that is never fit to image data at all. We propose CIRCLE, a class-incremental classifier built from fixed bidirectional two-dimensional reservoir features, adapted from BiRC2D for image classification, and streaming linear discriminant analysis heads. CIRCLE groups multiple random reservoir instantiations into feature ensembles and averages the softmax outputs of independent SLDA heads, yielding a tunable bias-variance tradeoff between richer random features and prediction-level ensembling. Because the feature extractor is fixed and the head admits streaming closed-form updates, CIRCLE performs sample-wise training without replay, task-boundary information, or backbone backpropagation. On CIFAR-100, TinyImageNet, ImageNet-Subset, and ImageNet-1k, CIRCLE is competitive at 10-20 task splits and substantially outperforms strong CS-EFCIL baselines at 50, 100, and 500 task splits, while training much faster than trained-backbone drift-compensation methods. Ablations show that the BiRC2D-style extractor, SLDA head, and balanced feature/prediction ensembling each contribute to the final performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。