用小批量数据加速贝叶斯神经网络推断，速度提升6倍且精度损失小。
Investigating Batch Inference in a Sequential Monte Carlo Framework for Neural Networks
- 在序贯蒙特卡洛框架中逐步引入小批量数据，实现高效采样。
- 在图像分类任务上达到6倍加速，准确率下降不足1%。
- 适合需要快速贝叶斯推断的深度学习应用，如不确定性建模。
贝叶斯推断可为通用神经网络（NN）的权重定义后验分布。精确后验通常不可计算，需采用近似方法。变分推断利用小批量随机梯度下降，计算高效，但依赖于与后验形式匹配的变分分布。基于粒子的方法如马尔可夫链蒙特卡洛和序贯蒙特卡洛（SMC）不假设后验的参数形式，但计算成本较高。这些采样方法通常使用全批量数据进行似然和梯度评估，导致高开销。本文探索在SMC采样器中逐步引入更多小批量数据（数据退火）的方法。结果表明，在基准图像分类任务上，可实现高达6倍的训练加速，同时保持精度损失极小。
原文摘要 · Abstract (English)
Bayesian inference allows us to define a posterior distribution over the weights of a generic neural network (NN). Exact posteriors are usually intractable, in which case approximations can be employed. One such approximation - variational inference - is computationally efficient when using mini-batch stochastic gradient descent as subsets of the data are used for likelihood and gradient evaluations, though the approach relies on the selection of a variational distribution which sufficiently matches the form of the posterior. Particle-based methods such as Markov chain Monte Carlo and Sequential Monte Carlo (SMC) do not assume a parametric family for the posterior by typically require higher computational cost. These sampling methods typically use the full-batch of data for likelihood and gradient evaluations, which contributes to this computational expense. We explore several methods of gradually introducing more mini-batches of data (data annealing) into likelihood and gradient evaluations of an SMC sampler. We find that we can achieve up to $6\times$ faster training with minimal loss in accuracy on benchmark image classification problems using NNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。