arXiv:2506.11030cs.LGcs.AI2025-06

用前向传递替代反向传播,实现高效低功耗神经网络训练

Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses

  • 用二次前向传递估算各层目标,无需反向传播和对称权重
  • 在MNIST/CIFAR上精度媲美反向传播,且在低精度下表现更优
  • 适合边缘设备与类脑计算,计算开销小,硬件友好

传统神经网络训练依赖反向传播(BP),虽成功但存在生物不真实、需对称反馈权重、信用分配非局部及反传时活动冻结等问题。本文提出前向目标传播(FTP),以二次前向传递替代反向传播,仅通过前馈计算估算各层目标,避免对称反馈权重和可学习逆函数,实现模块化与局部学习。我们在全连接网络、CNN和RNN上评估了FTP,在MNIST、CIFAR10和CIFAR100上达到与BP相当的准确率,并有效建模序列任务中的长期依赖。此外,FTP在量化低精度和新兴硬件约束下优于BP,显著优于其他类脑方法如目标传播变体和前向学习算法。其极低计算开销、纯前向特性及硬件兼容性,为节能型设备端学习和类脑计算提供了新方向。

原文摘要 · Abstract (English)

Training neural networks has traditionally relied on backpropagation (BP), a gradient-based algorithm that, despite its widespread success, suffers from key limitations in both biological and hardware perspectives. These include backward error propagation by symmetric weights, non-local credit assignment, and frozen activity during backward passes. We propose Forward Target Propagation (FTP), a biologically plausible and computationally efficient alternative that replaces the backward pass with a second forward pass. FTP estimates layerwise targets using only feedforward computations, eliminating the need for symmetric feedback weights or learnable inverse functions, hence enabling modular and local learning. We evaluate FTP on fully connected networks, CNNs, and RNNs, demonstrating accuracies competitive with BP on MNIST, CIFAR10, and CIFAR100, as well as effective modeling of long-term dependencies in sequential tasks. Moreover, FTP outperforms BP under quantized low-precision and emerging hardware constraints while also demonstrating substantial efficiency gains over other biologically inspired methods such as target propagation variants and forward-only learning algorithms. With its minimal computational overhead, forward-only nature, and hardware compatibility, FTP provides a promising direction for energy-efficient on-device learning and neuromorphic computing.

神经网络前向传播类脑计算低功耗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。