arXiv:2509.19063cs.LGcs.AI2025-09被引 1

提出新型训练算法MF,实现更高准确率与41%节能,挑战反向传播主导地位。

Beyond Backpropagation: Exploring Innovative Algorithms for Energy-Efficient Deep Neural Network Training

  • 设计无反向传播的单向前向算法MF,优化收敛路径提升泛化能力
  • 在相同硬件上比传统反向传播快34%、省电41%,碳排放显著降低
  • 揭示算法效率本质,适合关注绿色AI与硬件优化的研究者

深度神经网络(DNN)日益增长的计算与能耗需求主要源于反向传播(BP)机制,制约可持续人工智能发展。本文系统研究三种无反向传播训练方法:前向-前向(FF)、级联前向(CaFo)和单向前向(MF),从基础概念演进到性能更优的解决方案。构建了严谨对比框架:各算法在各自原生架构上实现(FF与MF用MLP,CaFo用CNN),并与其等效的BP模型进行比较。使用Optuna优化超参数,采用统一验证性能早停策略,确保模型充分调优后对比。结果表明,MF不仅在分类准确率上媲美甚至持续超越BP,且在验证损失曲面上收敛至更优极小值,挑战了“需全局优化才能达到顶尖性能”的假设。通过NVIDIA管理库(NVML)API进行硬件级测量,MF最高可降低41%能耗、缩短34%训练时间,按CodeCarbon估算碳足迹明显减少。此外,硬件分析揭示效率根源:暴露了FF的架构缺陷,验证了MF计算轻量设计的优势,并反驳了所有无反向传播方法均更省内存的普遍认知。本工作记录了从FF到MF的演进过程,为未来能源高效深度学习提供了清晰的数据驱动路线图。

原文摘要 · Abstract (English)

The rising computational and energy demands of deep neural networks (DNNs), driven largely by backpropagation (BP), challenge sustainable AI development. This paper rigorously investigates three BP-free training methods: the Forward-Forward (FF), Cascaded-Forward (CaFo), and Mono-Forward (MF) algorithms, tracing their progression from foundational concepts to a demonstrably superior solution. A robust comparative framework was established: each algorithm was implemented on its native architecture (MLPs for FF and MF, a CNN for CaFo) and benchmarked against an equivalent BP-trained model. Hyperparameters were optimized with Optuna, and consistent early stopping criteria were applied based on validation performance, ensuring all models were optimally tuned before comparison. Results show that MF not only competes with but consistently surpasses BP in classification accuracy on its native MLPs. Its superior generalization stems from converging to a more favorable minimum in the validation loss landscape, challenging the assumption that global optimization is required for state-of-the-art results. Measured at the hardware level using the NVIDIA Management Library (NVML) API, MF reduces energy consumption by up to 41% and shortens training time by up to 34%, translating to a measurably smaller carbon footprint as estimated by CodeCarbon. Beyond this primary result, we present a hardware-level analysis that explains the efficiency gains: exposing FF's architectural inefficiencies, validating MF's computationally lean design, and challenging the assumption that all BP-free methods are inherently more memory-efficient. By documenting the evolution from FF's conceptual groundwork to MF's synthesis of accuracy and sustainability, this work offers a clear, data-driven roadmap for future energy-efficient deep learning.

能量效率训练算法绿色AI无反向传播

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。