对比新型编码方式在神经形态记忆中的抗噪表现,发现编码与学习机制协同增效。
Rank-Order N-of-M Codes for Sparse Distributed Memory: Disentangling Representation and Learning Effects in Noise Robustness Against Contemporary Neuromorphic Architectures
- 采用排序型N选M编码替代传统二值编码,提升存储稳定性
- 在高负载下性能优于标准SDM达13.4个百分点
- 编码优势主要来自与MAX-Hebb学习的协同作用,适合持续学习系统设计
大型语言模型在持续学习方面仍受限,促使人们重新关注稀疏分布式记忆(SDM)作为显式在线情景记忆。CALM(Nechesov and Ruponen, 2025)将阈值二值编码视为开放问题。本文评估了排序型N-of-M编码(Furber et al., 2007)作为一种替代方案。贡献有三:第一,复现验证了原始架构,确认WheelSDM与RankOrderSDM在10个随机种子下余弦相似度为1.0000,且重现了RDLIF神经元在干扰下的偏差;第二,多种子容量实验显示,在缩放配置下,RankOrderSDM比StandardSDM在饱和状态高出13.4个百分点,而在原规模下也高出0.8个百分点;第三,误码率鲁棒性实验分离出表示与学习效应,表明显著鲁棒性提升主要源于排序编码与MAX-Hebbian学习的交互作用,而编码器本身在相同学习条件下仅带来微小优势。在GloVe-100嵌入上验证了这一微小但稳定的编码收益,而句子嵌入在低内存负载下已达到上限。次要分析显示,理想化排序编码在四比特精度下所需组件级编码能量仅为SpikingMamba的SI-LIF神经元的一半,尽管解码成本主导整体能耗。这些结果明确了原始排序型SDM架构中对现代记忆增强型AI系统具实际价值的组成部分,为CALM等架构提供实用指导。
原文摘要 · Abstract (English)
Large language models remain limited as continual learning systems, motivating renewed interest in Sparse Distributed Memory (SDM) as an explicit online episodic memory. CALM (Nechesov and Ruponen, 2025) identifies its threshold-binary encoder as an open design question. This paper evaluates rank-order N-of-M encoding (Furber et al., 2007) as an alternative. We make three contributions. First, a faithful reimplementation validates the published architecture by confirming exact equivalence between WheelSDM and RankOrderSDM (cosine similarity 1.0000 across 10 seeds) and reproducing the documented divergence of RDLIF neurons under interference. Second, multi-seed capacity experiments show RankOrderSDM outperforming StandardSDM by 13.4 percentage points at saturation in the scaled configuration and by 0.8 percentage points at the published architecture scale. Third, BER robustness experiments disentangle representation and learning effects, showing that the large robustness gain arises primarily from the interaction of rank-order encoding with MAX-Hebbian learning, while the encoder alone provides only a small advantage under matched learning conditions. Experiments on GloVe-100 embeddings confirm this small but consistent encoding benefit on real structured data, whereas sentence embeddings exhibit a ceiling effect at low memory load. A secondary analysis shows that idealized rank-order encoding requires half the component-level encoding energy of SpikingMamba's SI-LIF neurons at four-bit precision, although decoder costs dominate overall system energy. These results identify which components of the original rank-order SDM architecture provide measurable benefits for contemporary memory-augmented AI systems, offering practical guidance for architectures such as CALM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。