首个将前向-前向算法拓展至回归任务的框架,性能媲美反向传播。
FFR: Forward-Forward Learning for Regression

- 用分组竞争机制替代对比样本对,实现有序监督下的回归训练
- 平均恢复98.6%的反向传播精度,深度32时内存降至8%
- 支持多尺度预测与置信度估计,适合高可靠性回归场景
前向-前向(FF)算法通过局部、逐层优化提供一种计算高效且生物合理的反向传播替代方案。然而,FF原生设计用于分类任务,依赖正负样本对的对比学习,将其扩展到回归面临根本挑战:连续目标空间缺乏自然的“对立”样本,标准优度函数也无目标大小或排序信息。本文提出首个将FF拓展至真实世界回归任务的框架FFR。FFR引入三项关键创新:(1) 有序竞争优度函数,以距离感知的有序监督取代对比样本对,实现神经元组间的竞争学习;(2) 分层梯子架构,浅层学习粗粒度序判别,深层逐步细化为细粒度回归,并通过多尺度特征聚合促进层间协作;(3) 分层预测与不确定性估计,多尺度预测器联合输出鲁棒预测及预测置信度。大量实验表明,FFR在五个真实世界回归基准上平均恢复98.6%的反向传播精度,深度为8时峰值训练内存仅为其27%,深度为32时降至8%,单次迭代耗时约为反向传播的72%,显著优于所有无反向传播的现有方法。
原文摘要 · Abstract (English)
The Forward-Forward (FF) algorithm offers a computationally efficient and biologically plausible alternative to backpropagation (BP) by training neural networks through purely local, layer-wise optimization. However, FF is inherently designed for classification via contrastive positive-negative sample pairs, and extending it to regression poses fundamental challenges: continuous target space lack natural "opposites" for contrastive learning, and the standard goodness function carries no information about target magnitude or ordering. We propose FFR (Forward-Forward for Regression), to our knowledge, the first framework to extend FF to real-world regression and demonstrate competitive performance across diverse real-world datasets. FFR introduces three key innovations: (1) an ordinal competitive goodness function that replaces contrastive pairs with competitive learning between partitioned neuron groups under distance-aware ordinal supervision; (2) a stratified ladder architecture where shallow layers learn coarse ordinal discrimination and deeper layers refine into fine-grained regression, with multi-scale feature aggregation for inter-layer collaboration; and (3) hierarchical prediction with uncertainty estimation, where multi-scale predictors jointly provide robust predictions and prediction confidence as a free-lunch. Extensive experimental results show FFR recovers on average 98.6% of BP's accuracy across five real-world regression benchmarks while reducing peak training memory to only 27% of BP's at depth 8 and 8% at depth 32, with per-iteration time around 72% of BP's, and substantially outperforms all BP-free competitors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。