用优化视角重解预测编码，揭示神经网络与贝叶斯推理的统一机制。
Predictive Coding with Bayesian Priors via Proximal Gradients
- 将预测编码建模为连续时间近端梯度下降，统一优化与神经动力学
- 单层对应漏电放电率网络，层级结构由变量分裂松弛导出
- 适合研究神经计算、贝叶斯推断与类脑模型的读者
本文将预测编码重新表述为应用于正则化最大后验（MAP）目标的连续时间近端梯度下降。首先分析单层问题，发现近端梯度下降恰好对应一个漏电放电率网络：膜漏、有效递归矩阵、局部突触驱动和静态非线性均由同一优化原理导出，所得电路即为Rao与Ballard提出的模型。先验通过其近端算子决定非线性，似然精度设定观测增益。对于层级结构，经典深度MAP问题的变量分裂松弛生成了层次预测编码，作为局部与分布式求解器的互联。在概率建模中，该松弛将有向生成链替换为无向马尔可夫随机场，节点势函数为各层先验，每层应用自身激活函数，即其先验的近端算子。
原文摘要 · Abstract (English)
We recast predictive coding as continuous-time proximal gradient descent applied to a regularized maximum-a-posteriori (MAP) objective. We study first a single-level problem and then a multi-level hierarchy. For the single-level problem, we show that proximal gradient descent is precisely a leaky firing-rate network: the membrane leak, the effective recurrent matrix, the local synaptic drive, and the static nonlinearity all follow from one optimization principle, and the resulting circuit is the one proposed by Rao and Ballard. The prior selects the nonlinearity through its proximal operator, and the likelihood precision sets the gain on the observation. For the hierarchy, we show that a classical variable-splitting relaxation of the deep MAP problem yields hierarchical predictive coding as the interconnection of local and distributed solvers. In probabilistic modeling terms, this relaxation replaces the directed generative chain by an undirected Markov random field whose node potentials are the level-wise priors. Each level then applies its own activation function, namely the proximal operator of its prior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。