无需反向传播的前向算法在能效和速度上全面超越传统方法。
Energy-Efficient Deep Learning Without Backpropagation: A Rigorous Evaluation of Forward-Only Algorithms
- 提出无反向传播的单向前向算法（MF），直接优化前向传播结果。
- 在相同架构下，准确性更高，能耗降低41%，训练速度快34%。
- 适合追求高效、低功耗深度学习的硬件部署场景。
长期以来，反向传播（BP）被认为是实现顶尖性能的必要手段，但本文通过硬件验证的严谨实验挑战了这一观点。我们提出的单向前向（MF）算法是一种无需反向传播的方法，在其原生多层感知机（MLP）架构上，分类准确率持续高于经过最优调优的BP基线。该方法在泛化能力上表现更优，同时带来显著效率提升：能量消耗最多减少41%，训练速度最快提高34%。研究系统梳理了从格雷戈里·辛顿的前向-前向（FF）到级联前向（CaFo）再到MF的演进路径，并采用完全相同的网络架构与统一超参数优化框架进行公平比较。此外，我们还对无反向传播方法的内存效率进行了批判性重评，实证表明实际开销可能抵消理论优势。最终，本工作确立了MF作为MLP架构下兼具高性能、高效率与可持续性的实用替代方案。
原文摘要 · Abstract (English)
The long-held assumption that backpropagation (BP) is essential for state-of-the-art performance is challenged by this work. We present rigorous, hardware-validated evidence that the Mono-Forward (MF) algorithm, a backpropagation-free method, consistently surpasses an optimally tuned BP baseline in classification accuracy on its native Multi-Layer Perceptron (MLP) architectures. This superior generalization is achieved with profound efficiency gains, including up to 41% less energy consumption and up to 34% faster training. Our analysis, which charts an evolutionary path from Geoffrey Hinton's Forward-Forward (FF) to the Cascaded Forward (CaFo) and finally to MF, is grounded in a fair comparative framework using identical architectures and universal hyperparameter optimization. We further provide a critical re-evaluation of memory efficiency in BP-free methods, empirically demonstrating that practical overhead can offset theoretical gains. Ultimately, this work establishes MF as a practical, high-performance, and sustainable alternative to BP for MLPs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。