用生物约束网络模拟大脑多时标信息整合，实现零样本跨时标泛化。
Hierarchical temporal receptive windows and zero-shot timescale generalization in biologically constrained scale-invariant deep networks
- 基于海马体时间细胞构建分层时标网络，自然产生多尺度时间感受野。
- 模型参数减少数个数量级，且在未见时标上实现零样本泛化。
- 适合研究认知建模与具身智能的神经网络架构设计。
人类认知能整合嵌套的多时标信息。尽管皮层具有分层的时间感受野（TRWs），局部回路常表现出异质的时间常数。为调和这一矛盾，我们基于尺度不变的海马体时间细胞，在模仿语言层级结构（如‘字母’组成‘词’）的语言分类任务上训练了生物约束深度网络。首先，使用前馈模型（SITHCon），发现尽管各层内时间常数谱相同，但分层间仍自然涌现出层次化时间感受野。随后，将这些归纳偏置提炼为生物合理的循环架构（SITH-RNN）。通过训练从通用RNN到该受限子集的一系列架构，发现尺度不变的SITH-RNN以数个数量级更少的参数更快学习，并实现零样本跨分布时标泛化。结果表明，大脑采用尺度不变、序列化的先验——编码‘发生了什么’与‘何时发生’——使具备此类先验的循环网络特别适合描述人类认知。
原文摘要 · Abstract (English)
Human cognition integrates information across nested timescales. While the cortex exhibits hierarchical Temporal Receptive Windows (TRWs), local circuits often display heterogeneous time constants. To reconcile this, we trained biologically constrained deep networks, based on scale-invariant hippocampal time cells, on a language classification task mimicking the hierarchical structure of language (e.g., 'letters' forming 'words'). First, using a feedforward model (SITHCon), we found that a hierarchy of TRWs emerged naturally across layers, despite the network having an identical spectrum of time constants within layers. We then distilled these inductive priors into a biologically plausible recurrent architecture, SITH-RNN. Training a sequence of architectures ranging from generic RNNs to this restricted subset showed that the scale-invariant SITH-RNN learned faster with orders-of-magnitude fewer parameters, and generalized zero-shot to out-of-distribution timescales. These results suggest the brain employs scale-invariant, sequential priors - coding "what" happened "when" - making recurrent networks with such priors particularly well-suited to describe human cognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。