arXiv:2506.19726cs.LGcs.AI2025-06

让贝叶斯神经网络更精准,通过方向性不确定性建模提升校准能力。

Precise Bayesian Neural Networks

  • 在单位球面上用冯·米塞斯-费舍尔分布建模权重方向不确定性
  • 每层仅需一个可解释的噪声参数σ_eff,实现高效闭式计算
  • 适配归一化架构,无需牺牲精度即可改善不确定性校准

尽管历史久远,贝叶斯神经网络(BNN)与变分训练在实践中仍使用不足:标准高斯后验与网络几何不匹配,KL项在高维下易不稳定,且现有实现常增加复杂度却无法可靠提升不确定性估计。本文从归一化角度重审该问题:因归一化层消除了权重幅值影响,故仅在权重方向上建模不确定性,采用单位球面上的冯·米塞斯-费舍尔后验。高维几何导致每层仅需一个可解释标量——有效后归一化噪声σ_eff——其对应前向传播中的简单加性高斯噪声,并支持紧凑的、维度感知的闭式KL项。我们推导出浓度κ与激活方差、σ_eff之间跨不同场景的精确闭式近似,构建了一个轻量、可直接实现的变分单元,兼容现代归一化架构,在不损失准确率的前提下显著提升校准性能。维度感知对高维稳定优化至关重要。简言之,通过将变分后验与网络内在几何对齐,使BNN兼具原理性、实用性与精确性。

原文摘要 · Abstract (English)

Despite its long history, Bayesian neural networks (BNNs) and variational training remain underused in practice: standard Gaussian posteriors misalign with network geometry, KL terms can be brittle in high dimensions, and implementations often add complexity without reliably improving uncertainty. We revisit the problem through the lens of normalization. Because normalization layers neutralize the influence of weight magnitude, we model uncertainty \emph{only in weight directions} using a von Mises-Fisher posterior on the unit sphere. High-dimensional geometry then yields a single, interpretable scalar per layer--the effective post-normalization noise $σ_{\mathrm{eff}}$--that (i) corresponds to simple additive Gaussian noise in the forward pass and (ii) admits a compact, dimension-aware KL in closed form. We derive accurate, closed-form approximations linking concentration $κ$ to activation variance and to $σ_{\mathrm{eff}}$ across regimes, producing a lightweight, implementation-ready variational unit that fits modern normalized architectures and improves calibration without sacrificing accuracy. This dimension awareness is critical for stable optimization in high dimensions. In short, by aligning the variational posterior with the network's intrinsic geometry, BNNs can be simultaneously principled, practical, and precise.

贝叶斯神经网络不确定性建模归一化变分推断

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。