用梯度引导的采样器提升部分贝叶斯神经网络训练效率与性能。
Utilising Gradient-Based Proposals Within Sequential Monte Carlo Samplers for Training of Partial Bayesian Neural Networks
- 引入梯度驱动的提案机制与马尔可夫核，优化采样路径。
- 在预测精度和损失表现上超越现有最优方法。
- 支持大批次训练，显著缩短耗时且性能更优，适合大规模场景。
部分贝叶斯神经网络（pBNNs）仅对部分参数设为随机，却能与全贝叶斯神经网络竞争。采用序列蒙特卡洛（SMC）进行推断，可实现非参数化概率估计，优于传统参数方法。本文提出一种基于SMC的新训练方法，引入梯度引导提案并融合梯度驱动的马尔可夫核，提升高维问题下的可扩展性。实验表明，该方法在预测性能和最优损失上均优于当前最优水平。此外，pBNNs 在更大批量下表现出良好扩展性，大幅减少训练时间，常带来更优性能。
原文摘要 · Abstract (English)
Partial Bayesian neural networks (pBNNs) have been shown to perform competitively with fully Bayesian neural networks while only having a subset of the parameters be stochastic. Using sequential Monte Carlo (SMC) samplers as the inference method for pBNNs gives a non-parametric probabilistic estimation of the stochastic parameters, and has shown improved performance over parametric methods. In this paper we introduce a new SMC-based training method for pBNNs by utilising a guided proposal and incorporating gradient-based Markov kernels, which gives us better scalability on high dimensional problems. We show that our new method outperforms the state-of-the-art in terms of predictive performance and optimal loss. We also show that pBNNs scale well with larger batch sizes, resulting in significantly reduced training times and often better performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。