用牛顿更新改进神经网络集成，提升不确定性估计与收敛速度。
Stein Variational Newton Neural Network Ensembles
- 结合斯坦因变分牛顿法优化集成模型
- 训练轮次减少显著，后验分布更准确
- 适合需要可靠不确定性的高风险场景
深度神经网络集成是不确定性量化的重要工具，近期被重新从贝叶斯视角解释。然而，现有方法未能有效利用损失曲面的二阶信息，尽管高效的海森矩阵近似已可获得。本文提出一种新型近似贝叶斯推断方法，通过引入斯坦因变分牛顿更新，改造深度集成模型。该方法首次整合了可扩展的现代海森矩阵近似技术，实现更快收敛和更精准的后验分布逼近。在多种回归与分类任务中验证，相比现有基于集成的方法，本方法以显著更少的训练轮次取得更优性能，同时增强不确定性估计能力并提升抗过拟合鲁棒性。
原文摘要 · Abstract (English)
Deep neural network ensembles are powerful tools for uncertainty quantification, which have recently been re-interpreted from a Bayesian perspective. However, current methods inadequately leverage second-order information of the loss landscape, despite the recent availability of efficient Hessian approximations. We propose a novel approximate Bayesian inference method that modifies deep ensembles to incorporate Stein Variational Newton updates. Our approach uniquely integrates scalable modern Hessian approximations, achieving faster convergence and more accurate posterior distribution approximations. We validate the effectiveness of our method on diverse regression and classification tasks, demonstrating superior performance with a significantly reduced number of training epochs compared to existing ensemble-based methods, while enhancing uncertainty quantification and robustness against overfitting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。