采样法让贝叶斯神经网络更高效,该用它了。
Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learning
- 采样法与优化法计算效率相当,可替代传统方法
- 通过模型平均提升预测性能,揭示后验空间特征
- 适合追求不确定性量化与模型泛化能力的研究者
采样基推理(SAI)在贝叶斯神经网络(BNNs)中的实际应用仍受制于对采样可行性和效率的误解。本文认为,SAI已达到与优化方法相当的计算效率,正迈向超越后者成为有效、高效推理主流的临界点。这一进展应推动整个社区将BNN视为具备严谨不确定性量化能力的范式。采样法不仅能实现更优的预测性能(通过模型平均),还可作为众多下游任务的基础,并揭示BNN后验空间的关键特性。为实现这一转变,需首先破除现有误解,进而将研究重点转向两大核心挑战:充分探索后验空间,以及高保真地提炼后验样本以支持高效下游推理。克服这些概念与实践障碍,可释放采样法的全部潜力,使其成为贝叶斯深度学习的核心工具。
原文摘要 · Abstract (English)
The practical adoption of sampling-based inference (SAI) in Bayesian neural networks (BNNs) remains limited, partly due to persistent misconceptions about the feasibility and efficiency of sampling. This position paper argues that SAI has achieved computational parity with optimization-based methods and is at the verge of superseding such methods for effective and efficient inference in BNNs. This development should be in the interest of the whole community, promoting BNNs as a principled paradigm with its long-standing yet unfulfilled promise of providing principled uncertainty quantification for neural networks. SAI can even do more -- yielding superior prediction performance through model averaging, serving as the foundation for a plethora of possible downstream tasks, and providing crucial insights into the landscape of BNNs. In order to make such a change happen and unfold the potential of sampling, overcoming current misconceptions is a necessary first step. The next step is to realign research efforts toward addressing remaining challenges in SAI. In particular, the community must focus on two core problems: sufficient exploration of the posterior landscape and high-fidelity distillation of posterior samples for efficient downstream inference. By addressing conceptual and practical obstacles, we can unlock the full potential of SAI and establish it as a central tool in Bayesian deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。