提出新型前向-前向损失函数，提升模型对神经活动峰值的敏感性。
Selectivity and Shape in the Design of Forward-Forward Goodness Functions
- 设计基于峰值活动与分布形状的新型损失函数
- 在多个数据集上实现比传统方法高32.6个百分点的准确率
- 特别适合处理深层网络中激活值分布不均的问题
前向-前向（FF）算法通过局部‘优度函数’逐层训练网络，但目前仅研究了平方和（SoS）一种。本文系统探索优度函数的设计空间，发现核心原则：优度函数必须对神经活动的形状敏感，而非总能量。这一原则源于深度网络激活呈现重尾分布，且判别信息常集中在峰值活动。为此提出两类互补函数：选择性函数（top-k、entmax加权能量）仅关注峰值活动；形状敏感函数（超额峰度/“突发性”、高阶矩）通过尺度不变统计量奖励重尾分布。结合标签-特征分别前向传播（FFCL），在13种优度函数、5种激活函数、6个数据集及三组连续实验中，获得Fashion-MNIST 89.0%、MNIST（4x2000）98.2±0.1%的准确率，较SoS提升32.6个百分点，在所有基准上均有显著改进（USPS +72pp，SVHN +52pp）。突发性指标的尺度不变性使其对跨层与跨数据集的幅值变化具有更强鲁棒性。
原文摘要 · Abstract (English)
The Forward-Forward (FF) algorithm trains networks layer-by-layer using a local "goodness function," yet sum-of-squares (SoS) has remained the only choice studied. We systematically explore the goodness-function design space and identify a unifying principle: the goodness function must be sensitive to the shape of neural activity, not its total energy. This principle is motivated by the observation that deep network activations follow heavy-tailed distributions and that discriminative information is often concentrated in peak activities. We propose two complementary families: selective functions (top-k, entmax-weighted energy) that measure only peak activity, and shape-sensitive functions (excess kurtosis / "burstiness" and higher-order moments) that reward heavy-tailed distributions via scale-invariant statistics. Combined with separate label-feature forwarding (FFCL), controlled experiments across 13 goodness functions, 5 activations, 6 datasets, and three continuous sweeps, each tracing a characteristic inverted-U, yield 89.0% on Fashion-MNIST and 98.2+-0.1% on MNIST (4x2000), a +32.6pp gain over SoS, with consistent improvements across all benchmarks (+72pp USPS, +52pp SVHN). The scale-invariant nature of burstiness makes it particularly robust to magnitude shifts across layers and datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。