用抛物型方程约束损失变化，缓解持续学习中的遗忘问题。
Parabolic Continual Learning
- 将抛物型偏微分方程的边界条件引入记忆缓冲区，约束损失演化。
- 通过边界损失控制长期误差，有效减少遗忘和泛化偏差。
- 适合关注理论机制与稳定训练的持续学习研究者。
正则化持续学习方法对于预测算法在新数据分布下的行为至关重要。本文提出一种新方法，通过施加抛物型偏微分方程（PDE）的性质来正则化损失随时间的期望行为。这类抛物型PDE具备良好特性，可分析遗忘误差与泛化误差。具体而言，通过将记忆缓冲区作为边界施加边界条件，利用缓冲区约束长期依赖关系，使期望误差受边界损失限制。最后，我们在一系列持续学习任务上验证了该方法的实证性能。
原文摘要 · Abstract (English)
Regularizing continual learning techniques is important for anticipating algorithmic behavior under new realizations of data. We introduce a new approach to continual learning by imposing the properties of a parabolic partial differential equation (PDE) to regularize the expected behavior of the loss over time. This class of parabolic PDEs has a number of favorable properties that allow us to analyze the error incurred through forgetting and the error induced through generalization. Specifically, we do this through imposing boundary conditions where the boundary is given by a memory buffer. By using the memory buffer as a boundary, we can enforce long term dependencies by bounding the expected error by the boundary loss. Finally, we illustrate the empirical performance of the method on a series of continual learning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。