提出自适应正则化随机网络,解决无任务边界下的持续学习遗忘问题。
Randomized Neural Network with Adaptive Forward Regularization for Online Task-free Class Incremental Learning
- 用随机神经网络结合可调前向正则化,实现单次遍历的增量更新。
- 在两个图像数据集上6项指标均优于传统方法,有效避免灾难性遗忘。
- 支持自动调节正则强度,适合长期连续学习场景,无需人工调参。
类别增量学习(CIL)要求智能体在无任务边界提示的非独立同分布批次流中连续学习,并防止知识遗忘。现有方法在长序列学习中面临记忆衰退问题。为此,本文提出一种带前向正则化(-F)的随机神经网络框架,通过递归凸优化融合无监督知识,具备无学习衰减特性,性能优于经典岭回归风格(-R)。在此基础上,推导出可调前向正则化的集成深度随机向量函数链接网络(edRVFL-kF),k控制干预强度,支持闭式增量更新与可变学习率,避免重放和灾难性遗忘。为缓解非独立同分布带来的不稳定性及-kF的调参困难,进一步提出即插即用的edRVFL-kF-Bayes,使多子学习器的硬参数自动基于贝叶斯学习自适应确定。在2个图像数据集、6项评估指标、动态性能测试、消融实验与兼容性验证中,充分证明了-kF-Bayes与-kF框架的有效性。
原文摘要 · Abstract (English)
Class incremental learning (CIL) requires an agent to learn distinct tasks consecutively with knowledge retention against forgetting. Problems impeding the practical applications of CIL methods are twofold: (1) non-i.i.d batch streams and no boundary prompts to update, known as the harsher online task-free CIL (OTCIL) scenario; (2) CIL methods suffer from memory loss in learning long task streams, as shown in Fig. 1 (a). To achieve efficient decision-making and decrease cumulative regrets during the OTCIL process, a randomized neural network (Randomized NN) with forward regularization (-F) is proposed to resist forgetting and enhance learning performance. This general framework integrates unsupervised knowledge into recursive convex optimization, has no learning dissipation, and can outperform the canonical ridge style (-R) in OTCIL. Based on this framework, we derive the algorithm of the ensemble deep random vector functional link network (edRVFL) with adjustable forward regularization (-kF), where k mediates the intensity of the intervention. edRVFL-kF generates one-pass closed-form incremental updates and variable learning rates, effectively avoiding past replay and catastrophic forgetting while achieving superior performance. Moreover, to curb unstable penalties caused by non-i.i.d and mitigate intractable tuning of -kF in OTCIL, we improve it to the plug-and-play edRVFL-kF-Bayes, enabling all hard ks in multiple sub-learners to be self-adaptively determined based on Bayesian learning. Experiments were conducted on 2 image datasets including 6 metrics, dynamic performance, ablation tests, and compatibility, which distinctly validates the efficacy of our OTCIL frameworks with -kF-Bayes and -kF styles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。