arXiv:2507.21474cs.NEcs.AI2025-07被引 2

用类脑记忆机制提升RNN可解释性,实现与传统模型相当的性能

Hebbian Memory-Augmented Recurrent Networks: Engram Neurons in Deep Learning

  • 引入可微分显式记忆矩阵,结合海布学习规则构建动态记忆痕迹
  • 在三个基准上达到与LSTM/GRU相当的准确率和困惑度,尤其在WikiText-103表现稳定
  • 通过可视化揭示类生物记忆形成过程,适合关注模型可解释性的研究者

尽管在多种任务中取得成功,当前人工循环网络架构主要依赖隐式隐藏状态记忆,限制了其可解释性及对长程依赖的建模能力。相比之下,生物神经系统的显式关联记忆痕迹(即记忆印迹)通过海布突触可塑性强化,并在回忆时稀疏激活。受此神经生物学启发,我们提出新循环架构Engram Neural Network (ENN),包含可微分显式记忆矩阵、海布可塑性机制及稀疏注意力驱动的检索机制。该架构通过动态海布痕迹显式建模记忆形成与召回,相比传统RNN变体显著提升透明度与可解释性。我们在三个经典基准上评估:MNIST数字分类、CIFAR-10图像序列建模、WikiText-103语言建模。实验表明,ENN在准确性与泛化性能上与经典RNN、GRU、LSTM相当,在大规模WikiText-103任务中所有模型收敛至相似准确率与困惑度。同时,通过可观测的记忆动态显著提升可解释性;海布痕迹可视化进一步揭示类生物、结构化的记忆形成过程,验证了神经科学启发机制在构建更可解释、更鲁棒深度学习模型中的潜力。

原文摘要 · Abstract (English)

Despite success across diverse tasks, current artificial recurrent network architectures rely primarily on implicit hidden-state memories, limiting their interpretability and ability to model long-range dependencies. In contrast, biological neural systems employ explicit, associative memory traces (i.e., engrams) strengthened through Hebbian synaptic plasticity and activated sparsely during recall. Motivated by these neurobiological insights, we introduce the Engram Neural Network (ENN), a novel recurrent architecture incorporating an explicit, differentiable memory matrix with Hebbian plasticity and sparse, attention-driven retrieval mechanisms. The ENN explicitly models memory formation and recall through dynamic Hebbian traces, improving transparency and interpretability compared to conventional RNN variants. We evaluate the ENN architecture on three canonical benchmarks: MNIST digit classification, CIFAR-10 image sequence modeling, and WikiText-103 language modeling. Our empirical results demonstrate that the ENN achieves accuracy and generalization performance broadly comparable to classical RNN, GRU, and LSTM architectures, with all models converging to similar accuracy and perplexity on the large-scale WikiText-103 task. At the same time, the ENN offers significant enhancements in interpretability through observable memory dynamics. Hebbian trace visualizations further reveal biologically plausible, structured memory formation processes, validating the potential of neuroscience-inspired mechanisms to inform the development of more interpretable and robust deep learning models.

循环网络可解释性类脑计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。