arXiv:2410.07812cs.LGcs.AI2024-10NeurIPS被引 1

通过时序差分思想改进变分持续学习,缓解灾难性遗忘。

Temporal-Difference Variational Continual Learning

  • 引入多步后验正则化,减少递归更新中的误差累积。
  • 在多个基准上显著降低遗忘率,超越现有变分方法。
  • 适合需要长期稳定学习的部署场景,如智能系统迭代。

现实应用中的机器学习模型需持续学习新任务以适应数据分布变化。然而,持续学习(CL)模型常面临学习新任务(可塑性)与保留旧知识(记忆稳定性)之间的权衡,易导致灾难性遗忘,影响部署系统的可靠性。在贝叶斯持续学习中,变分方法通过递归更新后验分布并约束其接近先前估计来应对挑战。但本文指出,此类方法可能因递归过程中近似误差不断累积而失效。为此,我们提出新的学习目标,融合多个历史后验估计的正则化作用,防止单个误差主导后续更新,避免误差随时间放大。我们揭示了这些目标与强化学习及神经科学中时序差分(Temporal-Difference)方法的深刻联系。在多个具有挑战性的持续学习基准测试中,所提方法有效缓解了灾难性遗忘,性能优于强基线变分持续学习方法。

原文摘要 · Abstract (English)

Machine Learning models in real-world applications must continuously learn new tasks to adapt to shifts in the data-generating distribution. Yet, for Continual Learning (CL), models often struggle to balance learning new tasks (plasticity) with retaining previous knowledge (memory stability). Consequently, they are susceptible to Catastrophic Forgetting, which degrades performance and undermines the reliability of deployed systems. In the Bayesian CL literature, variational methods tackle this challenge by employing a learning objective that recursively updates the posterior distribution while constraining it to stay close to its previous estimate. Nonetheless, we argue that these methods may be ineffective due to compounding approximation errors over successive recursions. To mitigate this, we propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations, preventing individual errors from dominating future posterior updates and compounding over time. We reveal insightful connections between these objectives and Temporal-Difference methods, a popular learning mechanism in Reinforcement Learning and Neuroscience. Experiments on challenging CL benchmarks show that our approach effectively mitigates Catastrophic Forgetting, outperforming strong Variational CL methods.

持续学习变分推断时序差分灾难性遗忘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。