揭示扩散模型在零噪声下如何演变为关联记忆系统。
Associative Memory and Generative Diffusion in the Zero-noise Limit
- 用莫尔斯-斯梅尔动力系统统一建模记忆与生成机制。
- 证明零噪声极限下轨迹和不变测度保持稳定。
- 适用于能量模型、去噪扩散、霍普菲尔德网络等各类模型。
本文表明,生成式扩散过程在噪声趋近于零时收敛为关联记忆系统,并刻画了两类模型的稳定性、鲁棒性、记忆能力及生成动态。莫尔斯-斯梅尔动力系统被证明是关联记忆模型的通用逼近器,其白噪声扰动即为扩散过程。随着噪声减小,生成与记忆之间出现通用转变。莫尔斯-斯梅尔流的结构稳定性——即全局临界点结构的鲁棒性——意味着扩散过程在零噪声极限下的轨迹与不变测度同样稳定。这些模型的学习与生成景观表现为参数化的梯度流及其随机扰动,而莫尔斯-斯梅尔系统的分歧理论表明,它们在参数空间中除孤立值外普遍稳定,仅在可数的分岔点处发生局部或全局转变。这些景观因此具有有序的分歧序列,能创建、破坏或改变静点间的连接,并对微小随机或确定性扰动具有鲁棒性。该框架不依赖具体模型形式,通过能量模型、去噪扩散模型以及经典与现代霍普菲尔德网络实例验证。此外,我们推导出霍普菲尔德型网络的结构稳定性准则,发现简单情形常不满足。总体而言,这一几何方法为记忆与生成景观的分类、稳定性和涌现提供了深刻见解。
原文摘要 · Abstract (English)
This paper shows that generative diffusion processes converge to associative memory systems at vanishing noise levels and characterizes the stability, robustness, memorization, and generation dynamics of both model classes. Morse-Smale dynamical systems are shown to be universal approximators of associative memory models, with diffusion processes as their white-noise perturbations. The universal properties of associative memory that follow are used to characterize a generic transition from generation to memory as noise diminishes. Structural stability of Morse-Smale flows -- that is, the robustness of their global critical point structure -- implies the stability of both trajectories and invariant measures for diffusions in the zero-noise limit. The learning and generation landscapes of these models appear as parameterized families of gradient flows and their stochastic perturbations, and the bifurcation theory for Morse-Smale systems implies that they are generically stable except at isolated parameter values, where enumerable sets of local and global bifurcations govern transitions between stable systems in parameter space. These landscapes are thus characterized by ordered bifurcation sequences that create, destroy, or alter connections between rest points and are robust under small stochastic or deterministic perturbations. The framework is agnostic to model formulation, which we verify with examples from energy-based models, denoising diffusion models, and classical and modern Hopfield networks. We additionally derive structural stability criteria for Hopfield-type networks and find that simple cases violate them. Collectively, our geometric approach provides insight into the classification, stability, and emergence of memory and generative landscapes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。