arXiv:2605.08446cs.LG2026-05被引 1

放弃传统贝叶斯神经网络的后验逼近,改用预测得分直接优化信念分布。

Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks

论文配图:Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
图 1 · 摘自论文原文
  • 提出共享腔体目标函数,将后验约束变为可选,直接优化预测得分
  • 在7/8个UCI数据集上超越精确证据边界,且无需调参
  • 适用于小样本、嵌入迁移及端到端深度学习,适合追求校准与泛化者

标准贝叶斯深度学习训练目标以逼近后验为最终目标,但本文指出该目标是可移除的约束。基于Bethe/EP泛函的局部归一化结构,我们推导出一个共享腔体目标函数,其每条观测的数据项为任意似然下的严格正确预测评分。本文建议移除后验约束,采用自由路由训练信念分布——其先验与噪声超参数可在同一梯度步骤中学习。我们在高斯末层情形下实例化该方法(命名为SCROLL),实现单次通过,无需末层正则化权重交叉验证;其信念分布偏离后验存在闭式差距,由方差族表达的残余异方差性体现。SCROLL在八个UCI数据集中有七个优于精确证据边界,且在五成员集成预算下,预测似然和校准性匹配或超越验证调优基准,并可从UCI扩展至冻结嵌入与端到端深度学习。

原文摘要 · Abstract (English)

The standard training objectives of Bayesian deep learning are posterior-seeking: their optimum over the belief is the posterior of a fitted model, or its KL projection. We show that the shared target is a removable constraint on the belief-not an ideal that training can only approximate: posterior-seeking objectives define a binding map from model parameters to beliefs. From the local-normaliser structure of the Bethe/EP functional we derive a shared-cavity objective that carries this binding as an optional constraint, its per-observation data term a strictly proper predictive score for any likelihood. Our proposal is to drop the constraint. Free routing trains the belief as an optimisation variable of this objective; what trains on the predictive score is still a belief over weights, its prior and noise hyperparameters learned in the same gradient pass. We instantiate this at the Gaussian last layer, where exact inference is available: the freed belief departs from the posterior by a closed-form gap-the residual heteroscedasticity its variance family expresses. The instance, SCROLL, is single-pass, with no last-layer regularisation weight to cross-validate; it steps off the exact corner by a change of estimand (the shared cavity). SCROLL improves on the exact evidence corner at that corner's own learned features on seven of eight UCI datasets, matches or beats validation-tuned references on predictive likelihood and calibration-granted the ensembles' own five-member budget, it leads that tier on likelihood as well-and carries from UCI through frozen embeddings to end-to-end deep learning.

贝叶斯神经网络预测校准无后验优化单次训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。