提出ePC方法，让预测编码在数字硬件上更快更深地训练模型。
ePC: Fast and Deep Predictive Coding in Digital Simulation
- 用误差驱动重构的ePC新框架，避免信号指数衰减问题。
- 在多个模型和数据集上达到与反向传播相当的精度，深度模型表现更优。
- 适合追求高效训练、想探索类脑学习的算法研究者。
预测编码（PC）为神经网络训练提供了一种类脑替代方案，其本质是通过最小化系统内能实现。然而，在实际数字模拟中，传统基于状态的预测编码（sPC）因设计缺陷导致信号指数衰减，严重依赖计算资源且难以扩展至深层结构。本文揭示了该衰减机制的根本成因，并提出误差驱动的PC（ePC），一种重新参数化的框架，虽牺牲生物可解释性，但可精确计算梯度，且运行速度比sPC快数个数量级。在多种架构和数据集上的实验表明，ePC在深层模型中仍能逼近反向传播性能，显著优于sPC。本工作不仅带来实用提升，也为在数字硬件上实现更深层的预测编码学习提供了理论基础。
原文摘要 · Abstract (English)
Predictive Coding (PC) offers a brain-inspired alternative to backpropagation for neural network training, described as a physical system minimizing its internal energy. However, in practice, PC is predominantly digitally simulated, requiring excessive amounts of compute while struggling to scale to deeper architectures. This paper reformulates PC to overcome this hardware-algorithm mismatch. First, we uncover how the canonical state-based formulation of PC (sPC) is, by design, deeply inefficient in digital simulation, inevitably resulting in exponential signal decay that stalls the entire minimization process. Then, to overcome this fundamental limitation, we introduce error-based PC (ePC), a novel reparameterization of PC which does not suffer from signal decay. Though no longer biologically plausible, ePC numerically computes exact PC weights gradients and runs orders of magnitude faster than sPC. Experiments across multiple architectures and datasets demonstrate that ePC matches backpropagation's performance even for deeper models where sPC struggles. Besides practical improvements, our work provides theoretical insight into PC dynamics and establishes a foundation for scaling PC-based learning to deeper architectures on digital hardware and beyond.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。