arXiv:2507.14652stat.MLcs.CE2025-07被引 6

用变分推断筛选关键参数,加速神经网络的贝叶斯推断。

Accelerating Hamiltonian Monte Carlo for Bayesian Inference in Neural Networks and Neural Operators

  • 先用变分推断粗筛,找出影响预测不确定性的关键参数
  • 仅对关键参数运行高精度HMC,参数量减少90%以上
  • 适合需要可靠不确定性估计的大规模神经网络与物理模型

哈密顿蒙特卡洛(HMC)是贝叶斯推断中精确采样后验分布的强大方法,但在神经网络中因参数空间高维且后验非凸而计算成本高昂。现有近似方法如变分推断(VI)或随机梯度MCMC虽能降低开销,但会引入误差,导致不确定性估计不可靠。本文提出一种混合框架:先在全网络上进行低成本的VI训练,分析各参数对预测不确定性的贡献,发现大量参数对不确定性影响甚微。据此大幅压缩参数空间,仅对关键参数执行高精度全批HMC。该方法显著加速了大规模神经网络和神经算子的后验推断,可在含数十万参数的模型上高效运行。实验表明,该方法能有效学习复杂物理系统的代理模型,准确建模超音速流中上游条件到锥体壁面压力数据的映射关系。

原文摘要 · Abstract (English)

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network's parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. We show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

贝叶斯推断不确定性量化神经网络加速推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。