arXiv:2509.11285cs.LGcs.AI2025-09被引 1

不用反向传播,用数学公式直接更新分类器,实现高效稳定增量学习。

CIFNet: An Analytic Neural Learning Framework for Efficient and Calibrated Class-Incremental Learning

  • 基于闭式解的递归最小二乘法,无需反向传播更新分类器。
  • 在多个数据集上达到与迭代方法相当的准确率,且学习过程无性能骤降。
  • 适合追求低能耗、稳定训练的实时增量学习场景。

深度神经网络中的类增量学习(CIL)传统上依赖迭代梯度优化,导致计算开销大、超参数敏感且易灾难性遗忘。本文表明,利用冻结的预训练表征时,CIL可转化为一系列无需反向传播或迭代收敛的确定性闭式分类器适配问题。我们提出CIFNet,基于正则化递归最小二乘法(RRLS),在静态嵌入空间中通过精确的闭式岭回归解更新分类器权重。为缓解新增输出神经元因缺乏历史类别证据而产生的结构初始化偏差,CIFNet在隐空间引入轻量级校准缓冲区,并结合密度感知过采样,实现全局平衡的决策边界,无需存储原始图像或梯度更新。在CIFAR-100、ImageNet-100和CORe50上的大量实验表明,CIFNet在预测准确性上与迭代基线相当,同时保持严格单调、平滑的学习轨迹,无中间性能崩溃。此外,通过将逐轮反向传播替换为闭式矩累积,能量消耗最高降低20倍。这些结果确立了校准的解析学习作为神经网络持续适应的一种高效、稳定且数学严谨的新范式。

原文摘要 · Abstract (English)

Class-Incremental Learning (CIL) in deep neural networks is conventionally framed as an iterative gradient-based optimization problem, incurring high computational cost, hyperparameter sensitivity, and risk of catastrophic forgetting. In this work, we demonstrate that when leveraging frozen pre-trained representations, CIL can be solved as a sequence of deterministic, closed-form classifier adaptations without backpropagation or iterative convergence. We propose CIFNet, an analytic neural learning framework built upon Regularised Recursive Least-Squares (RRLS). CIFNet updates classifier weights via an exact, closed-form ridge-regression solution operating in a stationary embedding space. To counteract the structural initialisation bias that arises when newly expanded output neurons are introduced without exposure to past-class evidence, CIFNet incorporates a lightweight calibration buffer in latent space alongside density-aware oversampling, ensuring globally balanced decision boundaries without raw image storage or gradient updates. Extensive evaluations across CIFAR-100, ImageNet-100, and CORe50 show that CIFNet achieves predictive accuracy competitive with iterative CIL baselines while maintaining strictly monotonic, smooth learning trajectories free from intermediate performance collapse. Furthermore, by replacing epoch-wise backpropagation with closed-form moment accumulation, CIFNet achieves up to 20$\times$ reduction in energy consumption. These findings establish calibrated analytic learning as an efficient, stable, and mathematically grounded paradigm for continual adaptation in neural networks.

增量学习闭式解低能耗稳定训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。