提出在线贝叶斯堆叠法，提升模型组合的持续学习性能。
Bayesian Ensembling: Insights from Online Optimization and Empirical Bayes
- 从经验贝叶斯视角重看贝叶斯平均与堆叠，揭示其局限性。
- 设计OBS方法，通过优化对数评分动态融合模型，效果更优。
- 连接投资组合选择理论，提供可解释的算法与使用指导。
我们重新审视经典的贝叶斯集成问题，解决在在线、持续学习场景下如何学习最优贝叶斯模型组合的挑战。通过新的经验贝叶斯视角，我们重新解读了贝叶斯模型平均（BMA）和贝叶斯堆叠，揭示了BMA的固有缺陷。受在线优化启发，我们提出在线贝叶斯堆叠（OBS），通过优化预测分布的对数评分来自适应地组合模型。本工作关键贡献在于建立OBS与投资组合选择之间的新联系，将贝叶斯集成学习与成熟理论框架衔接，获得高效算法与详尽的遗憾分析。我们进一步厘清OBS与在线BMA的关系，表明二者优化不同但相关的损失函数。通过理论分析与实证评估，识别出OBS优于在线BMA的场景，并为实践者提供选择依据。
原文摘要 · Abstract (English)
We revisit the classical problem of Bayesian ensembles and address the challenge of learning optimal combinations of Bayesian models in an online, continual learning setting. To this end, we reinterpret existing approaches such as Bayesian model averaging (BMA) and Bayesian stacking through a novel empirical Bayes lens, shedding new light on the limitations and pathologies of BMA. Further motivated by insights from online optimization, we propose Online Bayesian Stacking (OBS), a method that optimizes the log-score over predictive distributions to adaptively combine Bayesian models. A key contribution of our work is establishing a novel connection between OBS and portfolio selection, bridging Bayesian ensemble learning with a rich, well-studied theoretical framework that offers efficient algorithms and extensive regret analysis. We further clarify the relationship between OBS and online BMA, showing that they optimize related but distinct cost functions. Through theoretical analysis and empirical evaluation, we identify scenarios where OBS outperforms online BMA and provide principled methods and guidance on when practitioners should prefer one approach over the other.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。