arXiv:2608.21307cs.LG2026-08

提出新方法缓解分析式持续学习中的旧类遗忘问题

SPARCL: Spectral Partitioned Analytic Continual Learning

论文配图:SPARCL: Spectral Partitioned Analytic Continual Learning
图 1 · 摘自论文原文
  • 将特征空间按能量分层,冻结高能主子空间的旧类分类器
  • 仅更新低能残差块,实现闭式更新且保证旧类输出不变
  • 在多个数据集上显著提升性能,适合追求稳定性的持续学习场景

分析式持续学习因其无需梯度优化而成为梯度类增量学习的有力替代方案,但即便使用精确递归求解器,旧类性能仍会下降。本文发现根源在于谱干扰:所有任务共享逆自相关算子 (R+λI)⁻¹,新任务样本若集中在旧任务主导特征方向,会稀释频谱并扰动旧类输出。为此提出 SPARCL,将运行自相关分解为高能量核心与残差补全部分,冻结核心子空间中的旧类分类器,仅通过递归最小二乘法更新残差块,并可选地引入残差随机投影扩展。该方法实现简单闭式更新,且对核心旧类输出具有可证明的不变性保证。在 CIFAR-100、CUB-200、ImageNet-R 与 ImageNet-A 上,基于 ViT-B/16 冻结协议,SPARCL 在性能上接近强表征匹配方法,同时与稀疏特征去相关方法(如 Fly-CL)互补。

原文摘要 · Abstract (English)

Analytic continual learning has emerged as a strong exemplar-free alternative to gradient-based class-incremental learning because it replaces iterative optimization with closed-form ridge updates. Yet the usual forgetting narrative, centered on stochastic gradient overwriting, does not explain why analytic methods still drift on old classes despite exact recursive solvers. We identify the culprit as spectral interference: the joint ridge classifier for all tasks shares the inverse autocorrelation operator $(R+λI)^{-1}$, so incoming task samples that load onto old dominant eigendirections dilute the spectrum and perturb old-class logits even when old labels are never revisited. Based on this view, we propose SPARCL, a spectral partitioned analytic continual learner that decomposes the running autocorrelation into a high-energy core and a residual complement, freezes old-class classifier components in the core subspace, and updates only the residual block through recursive least squares with an optional residual random-projection expansion. This yields a simple closed-form update with a provable invariance guarantee for the core contribution of old logits. Across CIFAR-100, CUB-200, ImageNet-R, and ImageNet-A under a frozen ViT-B/16 protocol, SPARCL closes most of the gap from classical analytic learners to strong representation matchers, while remaining complementary to sparse feature-decorrelation approaches such as Fly-CL.

持续学习谱分析分类器稳定ViT

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。