SSM模型在记忆任务中反而更记住开头数据，违背了理论预期。
Emergence of the Primacy Effect in Structured State-Space Models
- 在合成记忆任务中训练结构化状态空间模型
- 模型主要保留初始输入数据，呈现优先效应
- 揭示理论与实际的矛盾，适合研究模型记忆机制者关注
结构化状态空间模型（SSMs）旨在比传统循环神经网络具备更强的记忆保持能力，同时维持实时推理和解决Transformer的时间复杂度问题。尽管如此，经典SSMs的内存机制理论上应随时间单调衰减，即近期输入应被更准确地保留。然而，本研究在合成且统计平衡的记忆任务上训练和评估后发现，SSMs反而主要保留最初输入的数据，呈现出心理学中的“首因效应”（primacy effect）。这一反直觉现象挑战了当前对SSMs内存机制的理论理解，并为未来研究开辟了新方向。
原文摘要 · Abstract (English)
Structured state-space models (SSMs) have been developed to offer more persistent memory retention than traditional recurrent neural networks, while maintaining real-time inference capabilities and addressing the time-complexity limitations of Transformers. Despite this intended persistence, the memory mechanism of canonical SSMs is theoretically designed to decay monotonically over time, meaning that more recent inputs are expected to be retained more accurately than earlier ones. Contrary to this theoretical expectation, however, the present study reveals a counterintuitive finding: when trained and evaluated on a synthetic, statistically balanced memorization task, SSMs predominantly preserve the *initially* presented data in memory. This pattern of memory bias, known as the *primacy effect* in psychology, presents a non-trivial challenge to the current theoretical understanding of SSMs and opens new avenues for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。