测试一种类脑学习算法在深度网络中的表现，发现其训练不稳定且精度低于传统方法。
A Diagnostic Evaluation of Neural Networks Trained with the Error Diffusion Learning Algorithm
- 用全局误差驱动局部权重更新，实现无需反向传播的训练。
- 在多个数据集上精度均低于反向传播，深层网络中激活值会爆炸性增长。
- 引入RMS归一化缓解不稳定性，但无法弥补精度差距，仍需并行多网络结构。
误差扩散学习算法（EDLA）是一种通过单一全局误差信号驱动突触局部权重更新的学习机制。尽管最初被提出作为反向传播的替代方案，其行为尚未得到系统研究。本文提供了一种现代形式化与实现，并在奇偶校验、回归及图像分类任务（数字、MNIST、Fashion-MNIST、CIFAR-10）上评估了多层感知机使用EDLA的表现。依据原始设定，多类别分类通过为每类训练独立单输出网络实现，导致计算开销随类别数线性增长。在相似架构与训练协议下，EDLA在所有基准测试中均持续劣于反向传播基线。通过对内部动态的分析，发现基于ReLU的EDLA存在深度相关的失效模式：激活值可能爆炸性增长，引发训练不稳定和准确率下降。为缓解此问题，引入均方根归一化（RMSNorm），显著提升数值稳定性并扩展可训练深度范围，但未能弥合精度差距，且保留并行网络实现的开销。总体而言，本工作诊断出全局误差扩散在深度网络中失效的根源，为未来生物启发式学习规则的发展提供指导。
原文摘要 · Abstract (English)
The Error Diffusion Learning Algorithm (EDLA) is a learning scheme that performs synaptically local weight updates driven by a single, globally defined error signal. Although originally proposed as an alternative to backpropagation, its behavior has not been systematically characterized. We provide a modern formulation and implementation of EDLA and evaluate multilayer perceptrons trained with EDLA on parity, regression, and image-classification benchmarks (Digits, MNIST, Fashion-MNIST, and CIFAR-10). Following the original formulation, multi-class classification is implemented by training independent single-output networks (one per class), which makes the computational cost scale linearly with the number of classes. Under comparable architectures and training protocols, EDLA consistently underperforms backpropagation-trained baselines on all benchmarks considered. Through an analysis of internal dynamics, we identify a depth-related failure mode in ReLU-based EDLA: activations can grow explosively, causing unstable training and degraded accuracy. To mitigate this instability, we incorporate root mean square normalization (RMSNorm) into EDLA training. RMSNorm substantially improves numerical stability and expands the depth range in which EDLA can be trained, but it does not close the accuracy gap and retains the overhead of the parallel-network implementation. Overall, we offer a diagnostic evaluation of where and why global error diffusion breaks down in deep networks, providing guidance for future development of local, biologically inspired learning rules.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。