让记忆网络自动适应变化环境,不遗忘旧记忆还能动态扩容。
Associative Memory for Non-Stationary Environments: A Self-Sizing Generalization of Hopfield Networks
- 设计自适应记忆结构,随环境需求动态调整大小
- 无需预设容量,可在线收敛到最优存储规模
- 适合持续学习、记忆更新频繁的实时场景
霍普菲尔德网络虽是关联记忆的经典模型,但仅适用于静态环境——固定记忆集一次性存入冻结权重。真实环境是非平稳的(如记忆随时间到达、漂移、重复,需区分噪声),经典方法因灾难性干扰(橡皮擦问题)和容量预先固定而失效。我们提出非平稳环境中关联记忆的新范式:记忆与学习不再分离,适应性成为核心能力。本文提出一种自定规模的持续关联记忆模型,可无损存储新记忆、重绑定漂移与重复记忆,并仅对真正新颖内容分配新空间。其存储规模可在线收敛至环境的内在记忆需求,即乌里索恩宽度,通过对比相似性(CS)算子从数据中估计。实验表明,该模型在合成数据上实现了自定规模与持续保留,而非提升单条记忆召回精度,性能媲美强基线。
原文摘要 · Abstract (English)
The Hopfield network made associative memory (AM) the model system of neural computation, but it solves the problem only for a \emph{stationary} world: a fixed set of memories, stored once into frozen weights. Real environments are non-stationary (e.g., memories arrive over time, drift, recur, and must be told apart from noise), where the classical formulation fails by catastrophic interference (the palimpsest problem) and by a capacity fixed in advance. We argue that this is not a peripheral limitation but the crux: under non-stationarity, memory and learning cease to be separate problems, and \emph{adaptation}, rather than one-shot optimization, becomes the operative capacity. We give a fresh formulation of the AM problem for non-stationary environments and a \emph{self-sizing} continual associative memory that generalizes Hopfield's: it stores new memories without erasing old ones (no forgetting), re-binds drifting and recurring memories, allocates a genuinely new memory only for true novelty, and grows its store to the environment's \emph{intrinsic} memory demand and no further. We rigorously show that this demand is the Urysohn width of the problem and can be estimated from data via a contrastive-similarity (CS) operator. The memory's size converges to this capacity online with no preset value and no validation search, matching an oracle capacity search. We use experiments with synthetic datasets to show that the generalization buys self-sizing and retention under non-stationarity, \emph{not} higher per-item recall fidelity, on which it matches strong baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。