改进的反馈对齐算法在卷积网络中可模拟反向传播的表征结构
Biological Plausibility and Representational Alignment of Feedback Alignment in Convolutional Networks
- 用改进的反馈对齐训练卷积网络，权重更新机制更符合生物原理
- 在CIFAR-10上，其内部表征与反向传播结果结构相似
- 适合关注神经网络生物合理性与表征对齐的研究者
反馈对齐（FA）算法为神经网络训练提供了一种生物上更合理的替代反向传播（BP）的方法，但难以扩展到卷积架构。本文评估了包括改进型FA和标准BP在内的五种学习算法，在同一卷积网络和CIFAR-10数据集上的表现。通过生物合理性、可解释性和计算复杂度三方面对比分析发现，改进型FA算法在内部表示上与反向传播收敛于结构相似的表征。尤其表明，改进型FA的功能成功可能源于其对反向传播表征几何的模仿，尽管权重更新机制根本不同。
原文摘要 · Abstract (English)
The feedback alignment (FA) algorithm offers a biologically plausible alternative to backpropagation (BP) for training neural networks yet notably fails to scale to convolutional architectures. Modifications have been proposed to address this limitation, but at questionable cost to biological plausibility. In this paper, we evaluate five learning algorithms including modified FA and standard BP, applied to the same convolutional architecture with the CIFAR-10 dataset. We provide a tripartite comparative analysis focusing on biological plausibility, interpretability, and computational complexity. Our results indicate that modified FA algorithms converge on internal representations that are structurally similar to those produced by backpropagation. In particular, it appears the functional success of modified FA algorithms may be rooted in their ability to mimic the representational geometry of backpropagation, converging on similar representations despite relying on fundamentally different weight update mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。