用信号处理的数学框架，为持续学习提供理论支撑。
Mathematics of Continual Learning
- 借用力学与信号处理中的自适应滤波理论，构建持续学习的数学基础。
- 揭示两类方法在更新机制、记忆保持上的深层共性。
- 适合对理论推导和长期学习机制感兴趣的科研人员。
持续学习是机器学习中一个新兴领域，旨在让模型顺序学习多个任务而不遗忘先前知识。尽管已有大量基于深度学习的持续学习方法提出，但其数学基础仍不完善。另一方面，自适应滤波作为信号处理的经典领域，拥有严谨的数学方法体系，但在理解持续学习原理方面未被充分重视。本文综述了持续学习与自适应滤波的基本原理，并通过对比分析揭示二者间的多重关联。这些联系使得我们能够基于自适应滤波的已有成果强化持续学习的数学基础，同时借助持续学习的方法拓展自适应滤波的理解，并探讨由自适应滤波历史发展所启发的若干持续学习研究方向。
原文摘要 · Abstract (English)
Continual learning is an emerging subject in machine learning that aims to solve multiple tasks presented sequentially to the learner without forgetting previously learned tasks. Recently, many deep learning based approaches have been proposed for continual learning, however the mathematical foundations behind existing continual learning methods remain underdeveloped. On the other hand, adaptive filtering is a classic subject in signal processing with a rich history of mathematically principled methods. However, its role in understanding the foundations of continual learning has been underappreciated. In this tutorial, we review the basic principles behind both continual learning and adaptive filtering, and present a comparative analysis that highlights multiple connections between them. These connections allow us to enhance the mathematical foundations of continual learning based on existing results for adaptive filtering, extend adaptive filtering insights using existing continual learning methods, and discuss a few research directions for continual learning suggested by the historical developments in adaptive filtering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。