提升无反向传播的前向-前向学习，让卷积网络性能接近传统方法。
Covariance-Aware Goodness for Scalable Forward-Forward Learning
- 引入双轴协方差感知的损失函数，捕捉特征间空间与通道关联。
- 在ImageNet-100上达73.01%准确率，Tiny-ImageNet上达50.30%。
- 适合追求低内存、无梯度训练的模型设计者和硬件受限场景。
前向-前向算法消除了全局梯度流和全网络激活存储，但在卷积设置中，现有无反向传播方法在ImageNet-100和Tiny-ImageNet等复杂基准上显著落后于反向传播。我们识别出这一差距源于良度提取的结构性瓶颈：标准平方和形式将特征体积压缩为通道激活能量，忽略了关键的二阶依赖关系。为此，我们提出一个包含三个核心组件的框架：第一，双轴协方差良度（BiCovG）显式引入沿两个轴的结构化二阶信息——跨通道投影建模特征间协方差，嵌套多尺度聚合编码空间相关性统计；这在避免显式矩阵估计的O(C²)开销下，提供了可计算的协方差感知良度近似。第二，轻量级逻辑融合模块聚合层间预测，增强深层表示贡献。第三，特征对齐层（FAL）在块边界引入零初始化修正，缓解深度局部训练网络中的表示错位问题。通过这三个组件，我们使可行的前向-前向学习深度翻倍，将鲁棒层利用从浅层基线扩展至16层架构如VGG-16。所得无反向传播模型在ImageNet-100上达到73.01%，在Tiny-ImageNet上达到50.30%。作为实用扩展，混合良度块通过可配置块大小控制梯度传播范围，进一步将ImageNet-100差距缩小至3.6%，并在Tiny-ImageNet上匹配反向传播性能，同时相比反向传播降低约50%峰值内存。
原文摘要 · Abstract (English)
The Forward-Forward algorithm eliminates global gradient flow and full network activations storage. However, in convolutional settings, existing BP-free FF methods significantly under-perform backpropagation on complex benchmarks such as ImageNet-100 and Tiny-ImageNet. We identify this gap as a structural bottleneck in goodness extraction: standard sum-of-squares formulation collapses feature volumes into channel-wise activation energies which omits critical second-order dependencies. To address this, we propose a framework centered on three key components. First, Bi-axis Covariance Goodness(BiCovG) explicitly augments the standard goodness function with structured second-order information along two axes: cross-channel projections that model inter-feature covariance, and nested multi-scale aggregation that encodes spatial correlation statistics. This provides a tractable approximation to covariance-aware goodness without the prohibitive O(C^2) complexity of explicit matrix estimation. Second, a lightweight Logistic Fusion module aggregates layer-wise predictions, amplifying the contribution of deeper representations. Third, the Feature Alignment Layer(FAL) introduces a zero-initialized correction at block boundaries to mitigate representation misalignment in deep locally trained networks. By introducing these three components, we effectively double the depth of viable Forward-Forward learning, extending robust layer utilization from shallow baselines to 16 layer architectures like VGG-16. The resulting BP-free model achieves 73.01% on ImageNet-100 and 50.30% on Tiny-ImageNet. As a practical extension, Hybrid Goodness Blocks control the scope of gradient propagation via configurable block sizes, further narrowing the ImageNet-100 gap to 3.6% and matching BP on Tiny-ImageNet, while still reducing peak memory by approximately 50% relative to BP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。