提出新型消息传递框架，提升贝叶斯神经网络的不确定性量化能力。
Approximate Message Passing for Bayesian Neural Networks
- 构建因子图模型，用消息传递推断贝叶斯神经网络后验分布。
- 在CIFAR-10上性能媲美AdamW和IVON，校准性更优，可信区间相关性达0.9。
- 首次支持卷积网络且避免数据重复计算，适合高风险场景中的可信AI研究。
贝叶斯神经网络（BNNs）具有可靠不确定性量化与可解释性的潜力，对高风险领域可信AI至关重要。然而现有方法常面临过度自信、超参数敏感及后验坍缩等问题。本文推进了消息传递（MP）在BNN中的应用，提出首个能处理卷积神经网络且避免训练数据双重计数的因子图框架，克服了此前方法导致过度自信的缺陷。在含约89万参数的卷积网络上评估，于CIFAR-10上性能媲美当前最优基线AdamW和IVON，尤其在模型校准性方面表现更优。在合成数据上验证了不确定性估计的有效性，发现后验可信区间与覆盖真实生成函数的概率间存在0.9的强相关性。该方法已扩展至含560万参数的MLP，但尚需改进以达到先进变分推断方法的规模与性能水平。
原文摘要 · Abstract (English)
Bayesian neural networks (BNNs) offer the potential for reliable uncertainty quantification and interpretability, which are critical for trustworthy AI in high-stakes domains. However, existing methods often struggle with issues such as overconfidence, hyperparameter sensitivity, and posterior collapse, leaving room for alternative approaches. In this work, we advance message passing (MP) for BNNs and present a novel framework that models the predictive posterior as a factor graph. To the best of our knowledge, our framework is the first MP method that handles convolutional neural networks and avoids double-counting training data, a limitation of previous MP methods that causes overconfidence. We evaluate our approach on CIFAR-10 with a convolutional neural network of roughly 890k parameters and find that it can compete with the SOTA baselines AdamW and IVON, even having an edge in terms of calibration. On synthetic data, we validate the uncertainty estimates and observe a strong correlation (0.9) between posterior credible intervals and its probability of covering the true data-generating function outside the training range. While our method scales to an MLP with 5.6 million parameters, further improvements are necessary to match the scale and performance of state-of-the-art variational inference methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。