arXiv:2605.10970cond-mat.dis-nncs.AI2026-05

用上下文门控机制提升记忆检索效率,解释大模型零样本学习原理

Context-Gated Associative Retrieval: From Theory to Transformers

  • 引入上下文门控子电路,动态调整记忆检索能量景观
  • 理论证明可实现指数级检索性能提升,且存在唯一稳定解
  • 验证了大模型的上下文学习本质是上下文门控的记忆检索

霍普菲尔德网络及其推广形式建立了生物联想记忆、统计物理与变换器之间的深刻联系。然而,大多数模型将检索视为固定的查询到记忆映射,忽视了外部上下文在回忆中的作用。本文提出一种两阶段联想记忆架构,其中上下文门控子电路在回忆前和回忆过程中重塑检索能量景观。理论上证明,上下文门控能增强记忆间的分离度并诱导稀疏性,从而带来指数级的检索性能提升。关键的是,我们证明该系统存在唯一的自洽固定点,表明最终的检索状态由直接的上下文偏置和二级检索门反馈环共同驱动。随后,我们将这一理论与变换器相衔接;具体而言,在Llama-3上评估一阶近似,确认上下文学习表现为上下文门控的检索。原生动态符合理论:上下文定位记忆子空间,使零样本查询得以清晰区分。最终,该框架为联想记忆理论与大语言模型现象学提供了机制性连接。

原文摘要 · Abstract (English)

Hopfield networks and their generalizations have established deep connections among biological associative memories, statistical physics, and transformers. Yet most models treat retrieval as a fixed query-to-memory mapping, ignoring the role of external context in recall. In this work, we propose a two-stage associative memory architecture, wherein a context-gate subcircuit reshapes the retrieval energy landscape before and during recall. We show theoretically that context gating increases inter-memory separation while inducing sparsity, translating into exponential improvements in retrieval. Crucially, we prove that the system admits a unique self-consistent fixed point, revealing that the resulting retrieval state is driven by both a direct contextual bias and a second-order retrieval-gate feedback loop. We then bridge this theory to transformers; specifically, we evaluate a first-order approximation on Llama-3, confirming that in-context learning acts as context-gated retrieval. Native dynamics mirror our theory: context localizes a memory subspace, enabling the zero-shot query to cleanly discriminate. Ultimately, this framework provides a mechanistic link between associative memory theory and LLM phenomenology.

记忆检索大模型机制上下文学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。