提出贝叶斯预测编码,实现神经网络的不确定性量化与高效学习。
Bayesian Predictive Coding
- 用后验分布替代点估计,实现参数不确定性建模
- 全批量下收敛更快,小批量下性能不降
- 适合需要可信度评估的深度学习应用
预测编码(PC)是一种有生物学合理性的脑信息处理理论,通过变分自由能梯度下降优化隐状态和参数。但现有实现依赖最大后验(MAP)和最大似然(ML)估计,无法量化认知不确定性。本文提出贝叶斯预测编码(BPC),对网络参数进行后验分布估计。该方法保持了预测编码的局部性,导出闭式赫布型权重更新规则。在全批量设置下,BPC收敛速度优于传统PC;在小批量设置中仍具竞争力。此外,其不确定性量化能力可媲美主流贝叶斯深度学习方法,且具备更优的收敛特性。结果表明,BPC既是生物合理的脑学习机制,也是深度学习中极具吸引力的不确定性建模方案。
原文摘要 · Abstract (English)
Predictive coding (PC) is an influential theory of information processing in the brain, providing a biologically plausible alternative to backpropagation. It is motivated in terms of Bayesian inference, as hidden states and parameters are optimised via gradient descent on variational free energy. However, implementations of PC rely on maximum \textit{a posteriori} (MAP) estimates of hidden states and maximum likelihood (ML) estimates of parameters, limiting their ability to quantify epistemic uncertainty. In this work, we investigate a Bayesian extension to PC that estimates a posterior distribution over network parameters. This approach, termed Bayesian Predictive coding (BPC), preserves the locality of PC and results in closed-form Hebbian weight updates. Compared to PC, our BPC algorithm converges in fewer epochs in the full-batch setting and remains competitive in the mini-batch setting. Additionally, we demonstrate that BPC offers uncertainty quantification comparable to existing methods in Bayesian deep learning, while also improving convergence properties. Together, these results suggest that BPC provides a biologically plausible method for Bayesian learning in the brain, as well as an attractive approach to uncertainty quantification in deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。