让前向网络各层协同学习,提升模型效率与性能。
NetworkFF: Unified Layer Optimization in Forward-Only Neural Networks
- 通过层间协作机制,实现前向计算中的全局信息整合。
- 在MNIST和Fashion-MNIST上显著优于传统前向算法。
- 适合神经形态计算与低功耗AI系统应用。
前向-前向算法通过正负样本的双重前向传播,消除了反向传播带来的内存开销和生物不合理性。然而,传统实现存在层间隔离问题,各层独立优化,缺乏协同学习动态,限制了表征协调性并降低深层网络的收敛效率。本文提出协同前向-前向(CFF)学习框架,引入层间合作机制,在保持前向计算特性的前提下实现全局上下文融合。该框架包含两种协作范式:固定耦合(F-CFF)与可学习参数的自适应协作(A-CFF)。协同优度函数融合所有层的加权贡献,支持特征学习的协同优化,同时维持内存高效与生物合理性。在MNIST和Fashion-MNIST上的全面评估表明,CFF相比基线前向-前向方法取得显著性能提升。研究结果确立了层间协作作为前向-前向学习的核心改进方向,对神经形态计算架构和能效受限的AI系统具有直接应用价值。
原文摘要 · Abstract (English)
The Forward-Forward algorithm eliminates backpropagation's memory constraints and biological implausibility through dual forward passes with positive and negative data. However, conventional implementations suffer from critical inter-layer isolation, where layers optimize goodness functions independently without leveraging collective learning dynamics. This isolation constrains representational coordination and limits convergence efficiency in deeper architectures. This paper introduces Collaborative Forward-Forward (CFF) learning, extending the original algorithm through inter-layer cooperation mechanisms that preserve forward-only computation while enabling global context integration. Our framework implements two collaborative paradigms: Fixed CFF (F-CFF) with constant inter-layer coupling and Adaptive CFF (A-CFF) with learnable collaboration parameters that evolve during training. The collaborative goodness function incorporates weighted contributions from all layers, enabling coordinated feature learning while maintaining memory efficiency and biological plausibility. Comprehensive evaluation on MNIST and Fashion-MNIST demonstrates significant performance improvements over baseline Forward-Forward implementations. These findings establish inter-layer collaboration as a fundamental enhancement to Forward-Forward learning, with immediate applicability to neuromorphic computing architectures and energy-constrained AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。