揭示连续型记忆网络在几何约束下的容量极限与检索相变规律
Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory
- 基于球面几何推导连续神经元的熵，与核函数无关
- 零温下最大容量α=0.5，超过则出现检索/非检索临界线
- 不同核函数导致本质差异：一种支持完美检索，一种有噪声干扰
我们研究了现代霍普菲尔德网络（稠密关联记忆模型）在连续状态和几何约束下的热力学记忆容量，扩展了经典成对关联记忆分析。针对指数容量 M = e^{αN} 的情况，比较高斯（LSE）和埃潘尼科维奇（LSR）核函数。对于 N-球面上的连续神经元，几何熵仅取决于球面结构，与核函数无关。在尖锐核区域，零温时达到理论最大容量 α=0.5；低于该阈值时，存在一条临界线区分检索与非检索状态。两种核函数在相边界结构上定性不同：对于 LSE，所有 α>0 负载下均存在临界线；而对于 LSR，有限支撑引入阈值 αₜₕ，低于此值时无伪模式贡献噪声基底，且不存在临界线——任意温度下均可实现完美检索。这些结果推进了高容量关联记忆的理论，并阐明了类似注意力架构中检索鲁棒性的基本限制。
原文摘要 · Abstract (English)
We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical analyses of pairwise associative memory. We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity $M = e^{αN}$, comparing Gaussian (LSE) and Epanechnikov (LSR) kernels. For continuous neurons on an $N$-sphere, the geometric entropy depends solely on the spherical geometry, not the kernel. In the sharp-kernel regime, the maximum theoretical capacity $α= 0.5$ is achieved at zero temperature; below this threshold, a critical line separates retrieval from non-retrieval. The two kernels differ qualitatively in their phase boundary structure: for LSE, a critical line exists at all loads $α> 0$. For LSR, the finite support introduces a threshold $α_{\text{th}}$ below which no spurious patterns contribute to the noise floor, and no critical line exists -- retrieval is perfect at any temperature. These results advance the theory of high-capacity associative memory and clarify fundamental limits of retrieval robustness in modern attention-like memory architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。