arXiv:2410.02430cs.AIcs.CV2024-10NeurIPS被引 2

提出可生成多种未来可能的序列记忆模型,避免遗忘且在线学习。

Predictive Attractor Models

论文配图:Predictive Attractor Models
图 1 · 摘自论文原文
  • 通过神经元侧抑制实现在线持续学习,仅看一次输入即记下。
  • 能同时保留多个合理未来路径,生成多样预测结果。
  • 符合生物神经机制,适合研究认知科学与高效智能系统。

序列记忆是生物与人工智能的基础能力,支撑语言理解、规划和情景记忆等认知功能。现有方法存在灾难性遗忘、容量有限、学习缓慢、低阶马尔可夫记忆以及无法表示多重未来可能性等问题。受神经科学理论启发,本文提出预测吸引子模型(Predictive Attractor Models, PAM),一种新型序列记忆架构,具备生成能力。PAM 是流式模型,通过单次观察输入即可在线连续学习。其通过皮层微柱的侧抑制机制独特表征历史上下文,有效避免遗忘。模型通过吸引子结构采样生成未来预测,实现多路径生成。训练过程基于局部赫布可塑性规则,在生物合理框架下完成。论文还讨论了噪声鲁棒性、基于CPU的学习及容量扩展等优势。结果表明,PAM 在生物合理性与计算效率上均有显著提升,对认知科学与人工智能研究具有重要意义。

原文摘要 · Abstract (English)

Sequential memory, the ability to form and accurately recall a sequence of events or stimuli in the correct order, is a fundamental prerequisite for biological and artificial intelligence as it underpins numerous cognitive functions (e.g., language comprehension, planning, episodic memory formation, etc.) However, existing methods of sequential memory suffer from catastrophic forgetting, limited capacity, slow iterative learning procedures, low-order Markov memory, and, most importantly, the inability to represent and generate multiple valid future possibilities stemming from the same context. Inspired by biologically plausible neuroscience theories of cognition, we propose \textit{Predictive Attractor Models (PAM)}, a novel sequence memory architecture with desirable generative properties. PAM is a streaming model that learns a sequence in an online, continuous manner by observing each input \textit{only once}. Additionally, we find that PAM avoids catastrophic forgetting by uniquely representing past context through lateral inhibition in cortical minicolumns, which prevents new memories from overwriting previously learned knowledge. PAM generates future predictions by sampling from a union set of predicted possibilities; this generative ability is realized through an attractor model trained alongside the predictor. We show that PAM is trained with local computations through Hebbian plasticity rules in a biologically plausible framework. Other desirable traits (e.g., noise tolerance, CPU-based learning, capacity scaling) are discussed throughout the paper. Our findings suggest that PAM represents a significant step forward in the pursuit of biologically plausible and computationally efficient sequential memory models, with broad implications for cognitive science and artificial intelligence research.

序列记忆生成模型神经科学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。