提出新方法让神经网络训练更快更稳，适合硬件部署。
Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment
- 用直接反馈连接替代逐层传播，实现误差快速回传
- 理论误差传播时间从深度相关降至常数，提升训练效率
- 性能媲美甚至超越传统方法，适合定制化硬件加速
预测编码（PC）是一种受生物启发的神经网络训练算法，仅依赖局部更新，支持层间并行学习。但实际应用中存在两大瓶颈：误差信号需经多步前向推理从输出层逐层传递至浅层，且反馈在传播过程中呈指数衰减，导致浅层更新微弱。本文提出直接Kolen-Pollack预测编码（DKP-PC），同时解决反馈延迟与衰减问题，构建了一种更高效、可扩展的PC变体，且保持更新局部性。通过引入从输出层到各隐藏层的可学习反馈连接，建立误差传输的直达路径。该方法将理论误差传播时间复杂度从O(L)（L为网络深度）降低至O(1)，消除深度相关的延迟。实证结果表明，DKP-PC性能至少与标准PC相当，通常更优，且具备更低延迟和更好计算性能，具备面向专用硬件高效实现的潜力。
原文摘要 · Abstract (English)
Predictive coding (PC) is a biologically inspired algorithm for training neural networks that relies only on local updates, allowing parallel learning across layers. However, practical implementations face two key limitations: error signals must still propagate from the output to early layers through multiple inference-phase steps, and feedback decays exponentially during this process, leading to vanishing updates in early layers. We propose direct Kolen-Pollack predictive coding (DKP-PC), which simultaneously addresses both feedback delay and exponential decay, yielding a more efficient and scalable variant of PC while preserving update locality. Leveraging direct feedback alignment and direct Kolen-Pollack algorithms, DKP-PC introduces learnable feedback connections from the output layer to all hidden layers, establishing a direct pathway for error transmission. This yields an algorithm that reduces the theoretical error propagation time complexity from O(L), with L being the network depth, to O(1), removing depth-dependent delay in error signals. Moreover, empirical results demonstrate that DKP-PC achieves performance at least comparable to, and often exceeding, that of standard PC, while offering improved latency and computational performance, supporting its potential for custom hardware-efficient implementations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。