用生物合理机制提升卷积网络性能,达到与反向传播相当的准确率。
Advancing the Biological Plausibility and Efficacy of Hebbian Convolutional Neural Networks
- 融合硬获胜者通吃、高斯侧抑制和BCM规则构建生物可解释模型
- CIFAR-10上达75.2%准确率,比现有最优结果高10.6个百分点
- 适合关注神经机制真实性与高效学习的计算神经科学与AI研究者
本文系统探索了将赫布学习融入卷积神经网络(CNN)用于图像处理的架构,旨在提升其生物合理性。赫布学习基于局部无监督信息形成特征表示,是反向传播算法的替代方案,后者虽流行但被认为生物上不成立且计算成本高。所提出的最优架构整合了硬获胜者通吃(WTA)竞争、高斯侧抑制机制和Bienenstock-Cooper-Munro(BCM)学习规则,在单一模型中扩展了表征能力。在CIFAR-10测试后半段的平均分类准确率达75.2%,与端到端反向传播模型持平,且显著超越同深度网络下现有硬WTA方法的64.6%表现,提升10.6个百分点。在MNIST上达到98%准确率,STL-10上达69.5%。结果还显示,网络表现出稀疏分层学习特征,接收场逐渐变得复杂抽象。整体实现提升了表征性能与泛化能力,是迈向更生物合理的神经网络的重要一步。
原文摘要 · Abstract (English)
The research presented in this paper advances the integration of Hebbian learning into Convolutional Neural Networks (CNNs) for image processing, systematically exploring different architectures to build an optimal configuration, adhering to biological tenability. Hebbian learning operates on local unsupervised neural information to form feature representations, providing an alternative to the popular but arguably biologically implausible and computationally intensive backpropagation learning algorithm. The suggested optimal architecture significantly enhances recent research aimed at integrating Hebbian learning with competition mechanisms and CNNs, expanding their representational capabilities by incorporating hard Winner-Takes-All (WTA) competition, Gaussian lateral inhibition mechanisms, and Bienenstock-Cooper-Munro (BCM) learning rule in a single model. Mean accuracy classification measures during the last half of test epochs on CIFAR-10 revealed that the resulting optimal model matched its end-to-end backpropagation variant with 75.2% each, critically surpassing the state-of-the-art hard-WTA performance in CNNs of the same network depth (64.6%) by 10.6%. It also achieved competitive performance on MNIST (98%) and STL-10 (69.5%). Moreover, results showed clear indications of sparse hierarchical learning through increasingly complex and abstract receptive fields. In summary, our implementation enhances both the performance and the generalisability of the learnt representations and constitutes a crucial step towards more biologically realistic artificial neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。