提出自适应相似度机制,让记忆检索更准确。
Adaptive Hopfield Network: Rethinking Similarities in Associative Memory
- 基于生成视角重定义记忆检索,用后验概率衡量关联强度。
- 在噪声、掩码、偏差三种场景下理论证明最优正确率。
- 新模型在图像、表格、多实例学习等任务上表现领先。
关联记忆模型是内容可寻址的记忆系统,对生物智能至关重要且具有高可解释性。然而现有模型依赖距离衡量检索质量,无法保证返回与查询最强关联的模式,导致错误。本文提出查询是存储模式的生成变体,引入变体分布建模这一上下文相关的生成过程。因此,正确检索应返回使查询最可能源自该模式的后验概率最大值。此视角揭示理想相似度应近似各存储模式生成查询的似然,而现有固定相似度无法实现。为此,我们提出自适应相似度,通过从上下文中采样学习逼近这一未知似然,以实现正确检索。理论上证明该机制在三类典型变体(噪声、掩码、偏差)下达到最优正确检索。将其集成至新型自适应霍普菲尔德网络(A-Hop),实验表明其在记忆检索、表格分类、图像分类及多实例学习等任务中均达当前最优性能。
原文摘要 · Abstract (English)
Associative memory models are content-addressable memory systems fundamental to biological intelligence and are notable for their high interpretability. However, existing models evaluate the quality of retrieval based on proximity, which cannot guarantee that the retrieved pattern has the strongest association with the query, failing correctness. We reframe this problem by proposing that a query is a generative variant of a stored memory pattern, and define a variant distribution to model this subtle context-dependent generative process. Consequently, correct retrieval should return the memory pattern with the maximum a posteriori probability of being the query's origin. This perspective reveals that an ideal similarity measure should approximate the likelihood of each stored pattern generating the query in accordance with variant distribution, which is impossible for fixed and pre-defined similarities used by existing associative memories. To this end, we develop adaptive similarity, a novel mechanism that learns to approximate this insightful but unknown likelihood from samples drawn from context, aiming for correct retrieval. We theoretically prove that our proposed adaptive similarity achieves optimal correct retrieval under three canonical and widely applicable types of variants: noisy, masked, and biased. We integrate this mechanism into a novel adaptive Hopfield network (A-Hop), and empirical results show that it achieves state-of-the-art performance across diverse tasks, including memory retrieval, tabular classification, image classification, and multiple instance learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。