arXiv:2604.18857cs.LGcs.CV2026-04

通过分离学习与稳定机制,实现持续学习中不遗忘旧知识。

Task Switching Without Forgetting via Proximal Decoupling

论文配图:Task Switching Without Forgetting via Proximal Decoupling
图 1 · 摘自论文原文
  • 将学习和稳定性维护拆分为两个独立步骤,避免参数更新冲突。
  • 在标准基准上达到当前最优性能,无需重放或元学习组件。
  • 适用于需要高效利用模型容量的长期学习场景。

在持续学习中,主要挑战是学习新知识而不遗忘旧知识。现有方法通常通过正则化来惩罚对先前任务关键参数的改动,将正则项直接加入训练损失并用标准梯度下降优化,导致学习与保留信号混合,无法显式区分关键与冗余参数。随着任务序列增长,这种耦合会过度约束模型,限制前向迁移并降低容量利用率。本文提出一种基于算子分裂的新方法,将任务学习与稳定性强制分离:学习步骤专注于最小化当前任务损失,而近端稳定步骤采用稀疏正则化剪枝无关参数、保留任务相关参数。该方法将稳定性与可塑性转化为两个互补算子间的协商更新,而非冲突梯度。我们为该分裂方法提供了持续学习目标的理论支持,并证明所提求解器在标准基准上表现优异,实现了稳定性与适应性的双重提升,且无需重放缓冲区、贝叶斯采样或元学习组件。

原文摘要 · Abstract (English)

In continual learning, the primary challenge is to learn new information without forgetting old knowledge. A common solution addresses this trade-off through regularization, penalizing changes to parameters critical for previous tasks. In most cases, this regularization term is directly added to the training loss and optimized with standard gradient descent, which blends learning and retention signals into a single update and does not explicitly separate essential parameters from redundant ones. As task sequences grow, this coupling can over-constrain the model, limiting forward transfer and leading to inefficient use of capacity. We propose a different approach that separates task learning from stability enforcement via operator splitting. The learning step focuses on minimizing the current task loss, while a proximal stability step applies a sparse regularizer to prune unnecessary parameters and preserve task-relevant ones. This turns the stability-plasticity into a negotiated update between two complementary operators, rather than a conflicting gradient. We provide theoretical justification for the splitting method on the continual-learning objective, and demonstrate that our proposed solver achieves state-of-the-art results on standard benchmarks, improving both stability and adaptability without the need for replay buffers, Bayesian sampling, or meta-learning components.

持续学习正则化算子分裂

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。