无需反向传播，分层训练实现高效模型训练
Scalable Forward-Forward Algorithm
- 分层独立训练，避免反向梯度计算
- 在ResNet18上性能接近传统反向传播
- 模块化设计适合大模型，支持迁移学习
我们提出一种可扩展的前向-前向（Forward-Forward, FF）算法，通过逐层独立训练消除对反向传播的依赖。与传统反向传播不同，FF 避免了反向梯度计算，具有更好的模块化和内存效率，适用于大规模网络。我们通过引入卷积层的新损失计算方式，将 FF 扩展至 MobileNetV3 和 ResNet18 等现代卷积架构。实验表明，该方法性能与标准反向传播相当。进一步地，当将网络划分为块（如 ResNet 中的残差块），仅在块内使用反向传播而跨块不传梯度时，混合设计在保持相近训练速度的同时优于基线反向传播。最后，我们在小数据集和迁移学习任务上的实验验证了该方法的适应性。
原文摘要 · Abstract (English)
We propose a scalable Forward-Forward (FF) algorithm that eliminates the need for backpropagation by training each layer separately. Unlike backpropagation, FF avoids backward gradients and can be more modular and memory efficient, making it appealing for large networks. We extend FF to modern convolutional architectures, such as MobileNetV3 and ResNet18, by introducing a new way to compute losses for convolutional layers. Experiments show that our method achieves performance comparable to standard backpropagation. Furthermore, when we divide the network into blocks, such as the residual blocks in ResNet, and apply backpropagation only within each block, but not across blocks, our hybrid design tends to outperform backpropagation baselines while maintaining a similar training speed. Finally, we present experiments on small datasets and transfer learning that confirm the adaptability of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。