arXiv:2508.08762stat.MLcs.LG2025-08

提出基于预测编码的类脑神经网络,实现无需全局误差信号的局部权重更新。

Bio-Inspired Artificial Neural Networks based on Predictive Coding

  • 用预测编码替代反向传播,仅依赖局部神经活动更新权重
  • 理论支持梯度自动按不确定性缩放,提升训练稳定性
  • 适合对类脑计算、生物可解释性模型感兴趣的读者

反向传播(BP)是人工神经网络的核心训练算法,通过链式法则将损失梯度反向传播以最小化预测与目标输出之间的差异。然而,该方法要求每层权重更新依赖于网络输出端的全局误差信号,与生物大脑的局部可塑性机制不符。相比之下,预测编码(PC)作为一种生物合理替代方案,仅使用局部信息进行权重更新。其思想源自1950年代信号压缩研究,后被用于视觉皮层建模,并在自由能原理框架下形式化,与贝叶斯推断和动态系统建立关联。PC 的权重更新不依赖全局信号,具备梯度自动按不确定性缩放等理论优势。本文以教程形式介绍 PC 的数学推导、与现有优化及信号处理算法(如反向传播、卡尔曼滤波器)的联系,并提供基于 PyTorch 的 Python 实现示例,帮助读者从基础理解过渡到实际应用。

原文摘要 · Abstract (English)

Backpropagation (BP) of errors is the backbone training algorithm for artificial neural networks (ANNs). It updates network weights through gradient descent to minimize a loss function representing the mismatch between predictions and desired outputs. BP uses the chain rule to propagate the loss gradient backward through the network hierarchy, allowing efficient weight updates. However, this process requires weight updates at every layer to rely on a global error signal generated at the network's output. In contrast, the Hebbian model of synaptic plasticity states that weight updates are local, depending only on the activity of pre- and post-synaptic neurons. This suggests biological brains likely do not implement BP directly. Recently, Predictive Coding (PC) has gained interest as a biologically plausible alternative that updates weights using only local information. Originating from 1950s work on signal compression, PC was later proposed as a model of the visual cortex and formalized under the free energy principle, linking it to Bayesian inference and dynamical systems. PC weight updates rely solely on local information and provide theoretical advantages such as automatic scaling of gradients based on uncertainty. This lecture notes column offers a novel, tutorial-style introduction to PC, focusing on its formulation, derivation, and connections to well-known optimization and signal processing algorithms such as BP and the Kalman Filter (KF). It aims to support existing literature by guiding readers from the mathematical foundations of PC to practical implementation, including Python examples using PyTorch.

类脑计算预测编码神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。