用改进的采样方法让神经网络不再盲目自信，提升准确性与可靠性。
Humble your Overconfident Networks: Unlearning Overfitting via Sequential Monte Carlo Tempered Deep Ensembles
- 结合SGHMC的SMC算法支持小批量训练，实现高效贝叶斯推理。
- 在图像分类、异常检测等任务中优于SGD与深度集成模型。
- 可有效缓解过拟合，适合将预训练模型转为可信的贝叶斯模型。
序列蒙特卡洛（SMC）方法为贝叶斯不确定性量化提供了理论基础，但传统方法依赖全批量梯度计算，难以扩展。本文通过引入随机梯度哈密顿蒙特卡洛（SGHMC）作为提议机制，提出可扩展的SMCSGHMC算法，实现基于小批量的高效采样。该方法在图像分类、分布外检测（OOD）和迁移学习任务中均优于标准随机梯度下降（SGD）和深度集成模型。进一步实验表明，SMCSGHMC能有效缓解过拟合，改善模型校准性，为将预训练神经网络转化为具有良好校准性的贝叶斯模型提供灵活且可扩展的路径。
原文摘要 · Abstract (English)
Sequential Monte Carlo (SMC) methods offer a principled approach to Bayesian uncertainty quantification but are traditionally limited by the need for full-batch gradient evaluations. We introduce a scalable variant by incorporating Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) proposals into SMC, enabling efficient mini-batch based sampling. Our resulting SMCSGHMC algorithm outperforms standard stochastic gradient descent (SGD) and deep ensembles across image classification, out-of-distribution (OOD) detection, and transfer learning tasks. We further show that SMCSGHMC mitigates overfitting and improves calibration, providing a flexible, scalable pathway for converting pretrained neural networks into well-calibrated Bayesian models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。