用分层贝叶斯推断训练神经网络，提升模型不确定性估计能力。
BALI: Learning Neural Networks via Bayesian Layerwise Inference
- 将每层视为多元贝叶斯线性回归，利用前向输出和反向梯度定义伪目标。
- 层间后验为张量正态分布，协方差可克罗内克分解，支持高效求逆。
- 适用于小批量训练，收敛快，性能优于主流贝叶斯神经网络方法。
我们提出一种新的贝叶斯神经网络学习方法，将其视为一系列多变量贝叶斯线性回归的堆叠。核心思想是：若已知各层的目标输出，即可精确推断层间后验。我们以前向传播的层输出为基础，结合目标函数的反向传播梯度，定义伪目标。由此得到的层间后验服从张量正态分布，且协方差矩阵为克罗内克积分解形式，支持高效逆运算。该方法通过自然参数的指数移动平均扩展至随机小批量设置，实现对旧数据的渐进遗忘。算法在少数迭代内即收敛，在多种回归、分类及分布外检测任务上表现优于或相当主流贝叶斯神经网络方法。
原文摘要 · Abstract (English)
We introduce a new method for learning Bayesian neural networks, treating them as a stack of multivariate Bayesian linear regression models. The main idea is to infer the layerwise posterior exactly if we know the target outputs of each layer. We define these pseudo-targets as the layer outputs from the forward pass, updated by the backpropagated gradients of the objective function. The resulting layerwise posterior is a matrix-normal distribution with a Kronecker-factorized covariance matrix, which can be efficiently inverted. Our method extends to the stochastic mini-batch setting using an exponential moving average over natural-parameter terms, thus gradually forgetting older data. The method converges in few iterations and performs as well as or better than leading Bayesian neural network methods on various regression, classification, and out-of-distribution detection benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。