arXiv:2411.05861cs.NEcs.LG2024-11

不用反向传播和优化,用赫布学习实现手写数字识别

Rethinking Deep Learning: Non-backpropagation and Non-optimization Machine Learning Approach Using Hebbian Neural Networks

  • 用赫布学习规则替代反向传播训练神经网络
  • 在不依赖目标函数情况下达到约75%的准确率
  • 适合研究类脑计算与强人工智能的读者

开发强大人工智能可为社会与科学挑战提供有力工具。受生物系统启发的神经网络(NN)具备实现这一目标的潜力,但当前依赖误差反向传播的权重优化方法在生物系统中并未观测到,引发对现有神经网络方法的质疑。本文提出一种不使用目标函数、反向传播或优化过程的机器学习方法,通过在神经网络中实施赫布学习来解决MNIST分类问题,并分析其输出。研究分三阶段进行:第一阶段将赫布学习规则应用于伊东(2024)的MNIST识别算法,结果准确率低于非赫布网络,揭示传统训练流程对赫布学习的局限性;第二阶段通过基于范数的认知分析发现,单独训练于特定标签的神经网络对该标签响应强烈;第三阶段构建以向量范数大小为判据的识别程序,在无需大量数据集的情况下实现约75%的准确率。结果表明,赫布学习神经网络可在无目标函数、反向传播与优化的情况下识别手写字符。基于此,构建基于范数认知的机制并逐步增加复杂度以实现间接相似性认知,可能更贴近生物神经系统的运作,有助于推动强人工智能的发展。

原文摘要 · Abstract (English)

Developing strong AI could provide a powerful tool for addressing social and scientific challenges. Neural networks (NNs), inspired by biological systems, have the potential to achieve this. However, weight optimization techniques using error backpropagation are not observed in biological systems, raising doubts about current NNs approaches. In this context, Itoh (2024) solved the MNIST classification problem without using objective functions or backpropagation. However, weight updates were not used, so it does not qualify as machine learning AI. In this study, I develop a machine learning method that mimics biological neural systems by implementing Hebbian learning in NNs without backpropagation and optimization method to solve the MNIST classification problem and analyze its output. Development proceeded in three stages. In the first stage, I applied the Hebbian learning rule to the MNIST character recognition algorithm by Itoh (2024), resulting in lower accuracy than non-Hebbian NNs, highlighting the limitations of conventional training procedures for Hebbian learning. In the second stage, I examined the properties of individually trained NNs using norm-based cognition, showing that NNs trained on a specific label respond powerfully to that label. In the third stage, I created an MNIST character recognition program using vector norm magnitude as the criterion, achieving an accuracy of approximately 75%. This demonstrates that the Hebbian learning NNs can recognize handwritten characters without objective functions, backpropagation, optimization processes, and large data set. Based on these results, developing a mechanism based on norm-based cognition as a fundamental unit and then increasing complexity to achieve indirect similarity cognition should help mimic biological neural systems and contribute to realizing strong AI.

类脑计算赫布学习无监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。