用数学方法解决持续学习中遗忘与更新的矛盾。
Finding Structure in Continual Learning
- 将新旧知识学习解耦，通过迭代协商实现平衡。
- 无需外部记忆或复杂正则，稳定高效。
- 适合追求简洁高效持续学习系统的研究者。
从任务流中学习通常面临可塑性与稳定性之间的冲突：获取新知识常导致对旧信息的灾难性遗忘。现有方法多通过加权损失项求和来应对，引发梯度冲突，需借助外部记忆回放或参数正则化等复杂且低效策略。本文提出使用 Douglas-Rachford Splitting (DRS) 重述持续学习目标，将学习过程重构为两个解耦目标间的协商：一个促进新任务的可塑性，另一个维持旧知识的稳定性。通过其近端算子迭代寻找共识，DRS 提供了更原则化且稳定的动态机制。该方法在无需辅助模块或复杂附加组件的情况下，实现了稳定与可塑性的高效平衡，提供了一种更简单而强大的持续学习范式。
原文摘要 · Abstract (English)
Learning from a stream of tasks usually pits plasticity against stability: acquiring new knowledge often causes catastrophic forgetting of past information. Most methods address this by summing competing loss terms, creating gradient conflicts that are managed with complex and often inefficient strategies such as external memory replay or parameter regularization. We propose a reformulation of the continual learning objective using Douglas-Rachford Splitting (DRS). This reframes the learning process not as a direct trade-off, but as a negotiation between two decoupled objectives: one promoting plasticity for new tasks and the other enforcing stability of old knowledge. By iteratively finding a consensus through their proximal operators, DRS provides a more principled and stable learning dynamic. Our approach achieves an efficient balance between stability and plasticity without the need for auxiliary modules or complex add-ons, providing a simpler yet more powerful paradigm for continual learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。