用类海马结构模拟空间记忆,让智能体在稀疏视觉下也能走迷宫。
Emergence of Spatial Representation in an Actor-Critic Agent with Hippocampus-Inspired Sequence Generator
- 用生物启发的序列生成器构建时间记忆缓冲区,处理稀疏输入。
- 在16通道、约2.5%激活率下,性能优于LSTM,凸显稀疏性优势。
- 模型自动生成位置场与空间核,适合稀疏视觉导航任务研究者。
海马体位置细胞的序列放电通常被归因于沿轨迹的顺序感官驱动,也被认为与规划等认知功能有关。本文提出一种机制简洁的解释:海马序列源于内在的递归回路,可将短暂输入长期传播,作为时间记忆缓冲,在感官证据稀缺时尤为有用。我们设计了一个最小化序列生成器,受神经生物学启发,并与执行-评论家学习器结合,用于自我中心视觉导航。该智能体在无显式几何线索的连续迷宫中可靠求解,性能随递归序列长度提升。关键的是,在稀疏输入条件下(16通道,约2.5%活动率),模型表现优于LSTM,但在密集输入下则不然,揭示了表征稀疏性与记忆架构间的强交互作用。学习过程中,单元发展出局部位置场、距离依赖的空间核以及任务相关的重映射;序列生成器输入逐渐正交化,空间信息在各层递增。这些现象与神经生物学数据一致,且是性能的因果因素。结果表明,稀疏输入与序列生成动力学协同作用,既为哺乳动物海马体位置细胞序列提供了机制解释,也为基于稀疏自我中心输入的导航强化学习提供了一种简单归纳偏置。
原文摘要 · Abstract (English)
Sequential firing of hippocampal place cells is often attributed to sequential sensory drive along a trajectory, and has also been attributed to planning and other cognitive functions. Here, we propose a mechanistic and parsimonious interpretation to complement these ideas: hippocampal sequences arise from intrinsic recurrent circuitry that propagates transient input over long horizons, acting as a temporal memory buffer that is especially useful when reliable sensory evidence is sparse. We implement this idea with a minimal sequence generator inspired by neurobiology and pair it with an actor-critic learner for egocentric visual navigation. Our agent reliably solves a continuous maze without explicit geometric cues, with performance depending on the length of the recurrent sequence. Crucially, the model outperforms LSTM cores under sparse input conditions (16 channels, $\sim2.5\%$ activity), but not under dense input, revealing a strong interaction between representational sparsity and memory architecture. Through learning, units develop localized place fields, distance-dependent spatial kernels, and task-dependent remapping, while inputs to the sequence generator orthogonalize and spatial information increases across layers. These phenomena align with neurobiological data and are causal to performance. Together, our results show that sparse input synergizes with sequence-generating dynamics, providing both a mechanistic account of place cell sequences in the mammalian hippocampus and a simple inductive bias for reinforcement learning based on sparse egocentric inputs in navigation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。