提出一种类生物的双网络学习机制,解决传统反向传播的生物学合理性问题。
Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learning
- 用前馈与反馈双网络反向传递信号,模拟生物体内的逆流交换机制。
- 在多个数据集上表现接近传统方法,且无需权重传输。
- 适合研究神经科学启发的深度学习模型或可解释性算法设计者。
尽管误差反向传播在神经网络中广泛应用,但因其缺乏生物学合理性而受到质疑,如反向锁定和权重传输问题。为探索更符合生物神经系统的学习算法,我们受生物系统中逆流交换机制的启发,提出一种名为反向流学习(Counter-Current Learning, CCL)的信用分配框架。该框架采用前馈网络处理输入,反馈网络处理目标输出,两者通过反向信号传播相互增强。通过利用反馈网络底层更丰富的信息引导前馈网络顶层的更新,反之亦然,实现输入到输出的同步转换及动态双向影响。在MNIST、FashionMNIST、CIFAR10和CIFAR100数据集上,使用多层感知机与卷积神经网络进行实验,结果表明CCL性能与其它生物可解释算法相当,同时具备更强的生物学真实性。此外,我们展示了该方法在自编码器任务中的适用性,凸显其在无监督表示学习中的潜力。本工作为生物启发的深度学习算法提供了新方向。
原文摘要 · Abstract (English)
Despite its widespread use in neural networks, error backpropagation has faced criticism for its lack of biological plausibility, suffering from issues such as the backward locking problem and the weight transport problem. These limitations have motivated researchers to explore more biologically plausible learning algorithms that could potentially shed light on how biological neural systems adapt and learn. Inspired by the counter-current exchange mechanisms observed in biological systems, we propose counter-current learning (CCL), a biologically plausible framework for credit assignment in neural networks. This framework employs a feedforward network to process input data and a feedback network to process targets, with each network enhancing the other through anti-parallel signal propagation. By leveraging the more informative signals from the bottom layer of the feedback network to guide the updates of the top layer of the feedforward network and vice versa, CCL enables the simultaneous transformation of source inputs to target outputs and the dynamic mutual influence of these transformations. Experimental results on MNIST, FashionMNIST, CIFAR10, and CIFAR100 datasets using multi-layer perceptrons and convolutional neural networks demonstrate that CCL achieves comparable performance to other biologically plausible algorithms while offering a more biologically realistic learning mechanism. Furthermore, we showcase the applicability of our approach to an autoencoder task, underscoring its potential for unsupervised representation learning. Our work presents a direction for biologically inspired and plausible learning algorithms, offering an alternative mechanism of learning and adaptation in neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。