让卷积神经网络的前向传播更智能,提升无反向传播模型性能
Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignment

- 用可学习的通道-类别分配实现通道自适应分工
- 在多个数据集上超越现有前向算法,逼近反向传播效果
- 适合研究无反向传播训练机制的学者
前向-前向(FF)算法通过局部、仅前向的目标替代基于梯度的信用分配,为反向传播提供生物启发式替代方案。尽管已有研究将FF扩展至卷积神经网络(CNN),但现有方法依赖静态通道-类别划分,在复杂任务中表现受限。本文提出一种可学习的通道-类别分配机制,使卷积通道能根据数据自适应专化,并引入熵与正交性正则化以提升学习性能。此外,设计了一种基于损失感知的层贡献策略,依据中间层验证性能动态加权预测结果,增强仅前向推理的有效性。该方法集成于残差CNN,在CIFAR-10、CIFAR-100和Tiny-ImageNet上持续优于现有类似前向方法。尤其值得注意的是,其在基于FF的模型中建立新基准,显著缩小了与反向传播的差距。结果表明,引入可学习通道专化与层贡献加权,显著增强了深度CNN中仅前向学习的表征能力。
原文摘要 · Abstract (English)
The Forward-Forward (FF) algorithm offers a biologically inspired alternative to backpropagation by replacing gradient-based credit assignment with local, forward-only objectives. While recent extensions have adapted FF to convolutional neural networks (CNNs), existing formulations rely on static channel-class partitions and struggle to perform effectively in complex tasks. In this work, we introduce a learnable channel-class assignment mechanism that enables adaptive, data-driven specialization of convolutional channels, supported by entropy and orthogonality regularization to promote learning performance. We further propose a loss-aware layer contribution strategy that adaptively weights intermediate-layer predictions based on their validation performance, enhancing the effectiveness of forward-only inference. Integrated into residual CNNs, the proposed method achieves consistently superior performance across CIFAR-10, CIFAR-100, and Tiny-ImageNet compared to existing similar forward-only methods. Notably, it establishes new state-of-the-art performance among FF-based models, substantially narrowing the gap with backpropagation. These findings demonstrate that introducing learnable channel specialization and layer contribution weighting significantly enhances the representational capacity of forward-only learning in deep CNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。