提出自适应鲁棒的贝叶斯滤波方法，解决在线学习中的动态环境与高维参数挑战。
Adaptive, Robust and Scalable Bayesian Filtering for Online Learning
- 分模块设计实现在线学习的自适应性，应对非平稳环境变化。
- 引入广义贝叶斯框架，实现鲁棒滤波且计算开销与标准方法相当。
- 利用神经网络过参数化特性，高效更新高维参数，适合深度模型应用。
本论文将贝叶斯滤波作为解决多种序列机器学习问题的理论框架，涵盖在线（持续）学习、预序（一步预测）建模和上下文老虎机问题。针对在实际应用中面临的关键挑战——环境非平稳性下的适应性、模型误设与异常值的鲁棒性，以及深度神经网络高维参数空间的可扩展性，本文提出一系列新方法：(i) 模块化框架支持在线学习的自适应策略开发；(ii) 一种新型且可证明鲁棒的滤波器，计算成本与标准滤波器相当，采用广义贝叶斯；(iii) 一套基于近似二阶优化的方法，可顺序更新参数，充分利用高维参数模型（如神经网络）的过参数化特性。理论分析与实证结果表明，所提方法在动态、高维及模型误设场景下均表现出更优性能。
原文摘要 · Abstract (English)
In this thesis, we introduce Bayesian filtering as a principled framework for tackling diverse sequential machine learning problems, including online (continual) learning, prequential (one-step-ahead) forecasting, and contextual bandits. To this end, this thesis addresses key challenges in applying Bayesian filtering to these problems: adaptivity to non-stationary environments, robustness to model misspecification and outliers, and scalability to the high-dimensional parameter space of deep neural networks. We develop novel tools within the Bayesian filtering framework to address each of these challenges, including: (i) a modular framework that enables the development adaptive approaches for online learning; (ii) a novel, provably robust filter with similar computational cost to standard filters, that employs Generalised Bayes; and (iii) a set of tools for sequentially updating model parameters using approximate second-order optimisation methods that exploit the overparametrisation of high-dimensional parametric models such as neural networks. Theoretical analysis and empirical results demonstrate the improved performance of our methods in dynamic, high-dimensional, and misspecified models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。